Two-Factor Authentication Primer

I recently implemented two-factor authentication into a web app and since it was a new concept for me, I thought it would be good to explain the highest conceptual level of this process.  As with a lot of new things, there’s some terminology to learn and there’s a need to understand how all the pieces fit together.

First, what’s it take to integrate this with an existing profile login?  You need a new database field and a bit of extra code for opting in and out of the two-factor authentication.  Ideally, you’ll need a library for generating a QR code, too.

Before I get too much into it, these are some of the elements of the process.  There are three pieces of data involved:

  • Shared Secret: This element is stored in the database with the user profile and is never exposed outside your application.
  • Secret Key: This is an encoded version of the Shared Secret.  It is given to the user by your application and the user enters it into their authenticator application.
  • Code: The numeric value generated by the authenticator application.  This changes every minute.

In brief, your application and the authenticator application both use the current time plus the Secret Key to generate a Code.  If they match, the user is authenticated.

To implement this, you would modify your user profile page to provide a button to enable two-factor.  When the button is clicked, you create a a random Shared Secret and save it to their profile.  You use that Shared Secret to generate and return the Secret Key.  The user puts that Secret Key in their authenticator app and the opt-in is complete.

When the user logs in to your application, if they have a Shared Secret set in their profile, they are prompted to enter the Code from their authenticator app.  Your application compares that Code to the Code it generates itself, using the Secret Key (built from the Shared Secret).  If it is the same, the user is logged in.

It really is simple.  The only thing that isn’t clear, but can be found with some moderate Internet searching is the URL to embed in the QR code.  That URL is: otpauth://totp/{0}?secret={1}, where {0} is the name of the profile to use (either your application or the user’s username or both) and {1} is the Secret Key.  Authenticator apps allow manual entry of Secret Keys, so if you don’t provide a QR code, it’s still workable, just a bit tedious.

Some of the other pieces you’d need are functions to reset the Shared Secret or clear it, if the user wanted to opt-out.  This is simple user account maintenance.  With a simple implementation, you could blank out the Shared Secret on a “forgot password” action.  With more sensitive data, you may want a second code to allow a password reset.  The big concern is users who have lost their phone or wiped out their authenticator application entries.

Because two-factor authentication is so simple and is such a low-impact to existing user profile data structures (relative to oAuth), plus the fact it can be opt-in, it’s really a no-brainer to add it to your applications.

SSRS ReportViewer NullReferenceException on Dispose

I recently assisted on troubleshooting an error in a utility application where an exception was being thrown on the dispose of a Microsoft.Reporting.WebForms.ReportViewer.  The environmental conditions were pretty specific, so it’s possible you’d never see something like this in your environment.  But if you do, here’s how you can work around it.

The specific condition is that we have a shared library of code for both desktop and web applications.  One of the functions in that library takes some parameters for an SSRS report and returns a byte array for a rendered PDF of the report.  Because the library initially was used exclusively by the website, the WebForms version of the ReportViewer was used.  As time went on, the library was used by desktop apps and windows services.  That’s when the trouble began.

So, if you are using a WebForms.ReportViewer in a desktop application, you may get this exception when disposing the instance.  Digging into the decompiled code for the ReportViewer control suggested it was because there was no HttpContext available.  For us, the long-term fix was clear: use the WinForms version of the ReportViewer.  In the short term though, adding this line of code resolved the error:

If HttpContext.Current Is Nothing Then HttpContext.Current = New HttpContext(New HttpRequest(IO.Path.GetRandomFileName, "http://www.google.com", ""), New HttpResponse(IO.TextWriter.Null))

This created an HttpContext where there was none before, and the ReportViewer instance was able to be disposed without an error.

In Defense of Whitespace

There is a trend that I’ve been seeing recently that I find somewhat disturbing.  It primarily manifests itself with C# programmers, who also tend to be really indignant when questioned about it.  The issue is whitespace in source code.
When I see a big block of C# code and all the text is crammed into as little space as possible, it is very difficult to read.  This means that it takes longer to figure out what the code.  This means that it takes me longer to do my job, making me more expensive to my employer or client.  What benefit does this have with efficiency?  Further, although unrelated to this post, is the use of significant coding shortcut expressions in C#, which the developers praise as so efficient and elegant, but make the code barely intelligible.
Whenever I ask one of the coders about this, their answer is that whitespace is for people and compilers don’t need whitespace.  That response baffles me because it sounds like they are arguing my point.  The whole point of source code is to be human-readable.  But, somehow in their mind, it sounds like whitespace slows the application down.
Years ago, I read an excellent book, Developing User Interfaces for Microsoft Windows.  Although it’s rather outdated now, it had a lot of good advice in it, and one of the tips was to make use of whitespace for code clarity.  Up until that time, I didn’t pay much attention to blank lines and I had a different indenting scheme then what was the standard.  But then I changed both of these and my code became immediately more readable.
Although it maybe sounds a bit obvious, I demand whitespace in my code because I am  a writer and an avid reader.  I need paragraph breaks to indicate to me when a topic is changing or a new thought is starting.  If you treat writing a program like writing a story, your code will be much easier to understand; and to echo the C# developers, the compiler won’t care.
Aside from the line breaks between methods and between logical code blocks within methods, I like to put all my variable declarations at the beginning of the method.  It introduces you to all the characters in the chapter and gives you an idea of how complex the plotline of the chapter is.  This is also out of fashion with the current declare-just-before-use style.
One of my other structural designs that goes against the current fashion is to put my properties at the beginning of the class instead of at the end.  This is the same structure as a UML diagram, so I’m not sure why that design practice changed.  With methods, I try to put all my event handlers first, then order the methods by their access level (public, friend, protected, private).  Finally, I put methods that are called by other methods later in the class, so if you need to reference a called method, you almost always scroll down instead of up to find the called method.  This is made easier since private methods are last in the class.

Reading this post without whitespace is what it is like to read source code without whitespace.  It sucks.

They Thought They Could Stop Me.

I was working with a DataGrid that had a ButtonColumn in it.  I had a need to set the CommandArgument for this button.  Did you know there is no way to set a CommandArgument for a ButtonColumn?

image

I was all prepared to grab that control and set that property in the ItemDataBound event, but it doesn’t seem to exist.  Most people would resort to a template column, stick a button in it and work on that control.  Problem was, I was doing everything in code with no markup.  That adds a little complexity to that alternative.

Setting a simple breakpoint in the ItemDataBound event, I looked a little closer at what I had to work with in the Immediate window.

? e.Item.Controls(3)
{System.Web.UI.WebControls.TableCell}
    System.Web.UI.WebControls.TableCell: {System.Web.UI.WebControls.TableCell}
? e.Item.Controls(3).Controls.Count
1
? e.Item.Controls(3).Controls(0)
{Text = "Edit"}
    System.Web.UI.WebControls.DataGridLinkButton: {Text = "Edit"}

Hmm, it’s a DataGridLinkButton.  And it does have a CommandArgument property.  So let’s find that control and cast to that type and set that property.

image

I see.  So this type is not user-accessible.  It doesn’t even show up in the Object Browser.  However, it does show up in Reflector, and I can see that it inherits from LinkButton, which is public.  Let’s whip up a quick function to find that control and return a LinkButton for setting the CommandArgument.

Woah, slow down a bit.  This is a ButtonColumn and it can be a link button, command button, or an image button.  If we have a function specifically for LinkButton, it’s going to potentially error out.  In the typical, excellent design of the .NET framework, these three button types are all related using the IButtonControl interface, which has properties for CommandName and CommandArgument.  So by using the interface instead of the exact type, we’re being safe and future-proofing ourselves against other button types.

Private Function GetButtonColumnButton(row As DataGridItem, commandName As String) As IButtonControl
    Return RecurseRowControls(row, commandName)
End Function

Private Function RecurseRowControls(ctl As WebControl, commandName As String) As IButtonControl
    Dim btn As IButtonControl

    ' loop through embedded controls
    For Each c As WebControl In ctl.Controls
        btn = TryCast(c, IButtonControl)

        ' if it is a button and the command name matches, return it
        If btn IsNot Nothing AndAlso String.Compare(btn.CommandName, commandName, True) = 0 Then
            Return btn
        End If

        ' if the control has child control, search them for the button
        If c.HasControls Then
            btn = RecurseRowControls(c, commandName)
            If btn IsNot Nothing Then Return btn
        End If
    Next

    ' no button found
    Return Nothing

End Function

And just like that, we can now have access to the button’s properties like Text, CommandName, CommandArgument, and CausesValidation.  That’s some great stuff there.

Private Sub Grid_ItemDataBound(sender As Object, e As DataGridItemEventArgs) Handles Me.ItemDataBound
    Dim btn As IButtonControl

    If e.Item.ItemType = ListItemType.AlternatingItem Or e.Item.ItemType = ListItemType.Item Then
        btn = GetButtonColumnButton(e.Item, "Edit")
        btn.CommandArgument = "something like an ID"
        btn.Text = "specific text label"
    End If
End Sub

Saving Objects–Simple, Not Difficult

XML can be a wonderful thing.  Storing XML in SQL Server can be wonderful, too.  It gives you a place to store a lot of data in one field and is especially useful if that data is considered one unit of data.  A good example of this would be storing user preferences, for example, a stripped down table like:

image

So that’s one field: preferences.  Now let say that we have some items we want to store, like a few general preferences and some dialog boxes where the user checked the “Do not show this message again” option:

Public Class UserPreferences

    Private _UserID As Integer
    Private _General As GeneralPreferences
    Private _DoNotShowMessages As DoNotShowMessagesPreferences

    Public Property General As GeneralPreferences
        Get
            If _General Is Nothing Then _General = New GeneralPreferences
            Return _General
        End Get
        Set(ByVal value As GeneralPreferences)
            _General = value
        End Set
    End Property

    Public Property DoNotShowMessages As DoNotShowMessagesPreferences
        Get
            If _DoNotShowMessages Is Nothing Then _DoNotShowMessages = New DoNotShowMessagesPreferences
            Return _DoNotShowMessages
        End Get
        Set(ByVal value As DoNotShowMessagesPreferences)
            _DoNotShowMessages = value
        End Set
    End Property

    Public Sub New()

    End Sub

    Private Sub New(ByVal userID As Integer)
        _UserID = userID
    End Sub

    Public Class GeneralPreferences
        Public Property ShowSplashScreen As Boolean
        Public Property UseAlternateColorScheme As Boolean
        Public Property NumberOfItemsinGrids As Integer
    End Class

    Public Class DoNotShowMessagesPreferences
        Public Property HideNoResultsMessage As Boolean
        Public Property HideCloseConfirmation As Boolean
    End Class
End Class

This gives us two nested classes that store our values in nice groups, held in a class that allows us to access those nested classes and set the values.  Now we want to create an XML document that we can save and load in SQL.  So saving would be something like:

    Public Sub SaveXML()
        Dim doc As Xml.XmlDocument
        Dim parentNode As XmlNode
        Dim childNode As XmlNode
        Dim parms As New Generic.List(Of SqlClient.SqlParameter)

        doc = New XmlDocument
        doc.LoadXml("<UserPreferences />")

        parentNode = doc.DocumentElement.AppendChild(doc.CreateNode(XmlNodeType.Element, "General", doc.NamespaceURI))

        childNode = parentNode.AppendChild(doc.CreateNode(XmlNodeType.Element, "ShowSplashScreen", doc.NamespaceURI))
        childNode.InnerText = Me.General.ShowSplashScreen.ToString

        childNode = parentNode.AppendChild(doc.CreateNode(XmlNodeType.Element, "UseAlternateColorScheme", doc.NamespaceURI))
        childNode.InnerText = Me.General.UseAlternateColorScheme.ToString

        childNode = parentNode.AppendChild(doc.CreateNode(XmlNodeType.Element, "NumberOfItemsInGrids", doc.NamespaceURI))
        childNode.InnerText = Me.General.NumberOfItemsInGrids.ToString

        parentNode = doc.DocumentElement.AppendChild(doc.CreateNode(XmlNodeType.Element, "DoNotShowMessages", doc.NamespaceURI))

        childNode = parentNode.AppendChild(doc.CreateNode(XmlNodeType.Element, "HideNoResultsMessage", doc.NamespaceURI))
        childNode.InnerText = Me.DoNotShowMessages.HideNoResultsMessage.ToString

        childNode = parentNode.AppendChild(doc.CreateNode(XmlNodeType.Element, "HideCloseConfirmation", doc.NamespaceURI))
        childNode.InnerText = Me.DoNotShowMessages.HideCloseConfirmation.ToString

        With parms
            .Add(New SqlClient.SqlParameter("@UserID", _UserID))
            .Add(New SqlClient.SqlParameter("@Preferences", doc.OuterXml))
        End With

        SqlHelper.ExecuteNonQuery(CONN_STRING, CommandType.Text, _
            "insert userpreferences(userid,preferences) values(@UserID,@Preferences)", parms.ToArray)

    End Sub

and would give us an xml document saved to the server like:

<UserPreferences>
  <General>
    <ShowSplashScreen>True</ShowSplashScreen>
    <UseAlternateColorScheme>False</UseAlternateColorScheme>
    <NumberOfItemsInGrids>3</NumberOfItemsInGrids>
  </General>
  <DoNotShowMessages>
    <HideNoResultsMessage>False</HideNoResultsMessage>
    <HideCloseConfirmation>True</HideCloseConfirmation>
  </DoNotShowMessages>
</UserPreferences>

We’d also need a corresponding LoadXML method to read and parse out the XML and set the internal values.  That seems pretty good and it prevents us from having to modify the database table every time we add a new preference.

But, in a more critical and more annoying way, we have to modify the SaveXML and LoadXML methods every time we add a new preference.  Not only that, but we have to compensate for previously-saved versions of the XML, saved before new preferences were added, otherwise we’ll get errors when we try to read nodes that don’t exist.  This is a path of misery and spaghetti.

Don’t be discouraged.  There is an easy way.  In fact, it’s so easy it’s near unbelievable.  You add this code once and never change it.  Add all the preferences/properties you want and the code works with whatever is available.  It uses the XMLSerializer to do all the work.

First, all the classes need to be marked as <Serializable()>.  Then we create a method to instantiate the UserPreferences class:

    Shared Function GetInstance(ByVal userID As Integer) As UserPreferences
        Dim up As UserPreferences
        Dim ser As XmlSerializer
        Dim dt As DataTable
        Dim dr As DataRow

        dt = SqlHelper.ExecuteDataset(CONN_STRING, CommandType.Text, _
            "select preferences from userpreferences where userid=" & userID).Tables(0)

        If dt.Rows.Count <> 0 Then
            dr = dt.Rows(0)
            ser = New XmlSerializer(GetType(UserPreferences))
            up = TryCast(ser.Deserialize(New IO.StringReader(CStr(dr("Preferences")))), UserPreferences)
            up._UserID = userID

        Else
            up = New UserPreferences(userID)

        End If

        dt.Dispose()

        Return up

    End Function

Then we add a public method to save the class:

    Public Sub Save()
        Dim ser As XmlSerializer
        Dim xmlData As IO.StringWriter
        Dim parms As New Generic.List(Of SqlClient.SqlParameter)

        ser = New XmlSerializer(GetType(UserPreferences))
        xmlData = New IO.StringWriter
        ser.Serialize(xmlData, Me)

        With parms
            .Add(New SqlClient.SqlParameter("@UserID", _UserID))
            .Add(New SqlClient.SqlParameter("@Preferences", xmlData.GetStringBuilder.ToString))
        End With

        SqlHelper.ExecuteNonQuery(CONN_STRING, CommandType.Text, _
            "insert userpreferences(userid,preferences) values(@UserID,@Preferences)", parms.ToArray)

    End Sub

Seriously, that’s it.  Three lines to turn an object into XML.  Two lines to create an object from XML.  Missing and extraneous properties get skipped with no errors, which lets you change the classes whenever and however you need.  And the structure of the XML is the same as shown previously, with child classes/properties as nested elements.

In a test app, you can load, change, and save preferences:

image

With the preferences in an object, the UI code is extremely simple – one of the great benefits to using objects:

    Dim _prefs As UserPreferences

    Private Sub cmdLoad_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles cmdLoad.Click
        _prefs = UserPreferences.GetInstance(CInt(txtUserID.Text))

        With chkGeneralPreferences
            .SetItemChecked(0, _prefs.General.ShowSplashScreen)
            .SetItemChecked(1, _prefs.General.UseAlternateColorScheme)
        End With

        txtNumberOfItemsInGrid.Value = _prefs.General.NumberOfItemsInGrids

        With chkDoNotShowMessages
            .SetItemChecked(0, _prefs.DoNotShowMessages.HideNoResultsMessage)
            .SetItemChecked(1, _prefs.DoNotShowMessages.HideCloseConfirmation)
        End With

    End Sub

    Private Sub cmdSave_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles cmdSave.Click
        With _prefs.General
            .ShowSplashScreen = chkGeneralPreferences.GetItemCheckState(0) = CheckState.Checked
            .UseAlternateColorScheme = chkGeneralPreferences.GetItemCheckState(1) = CheckState.Checked
        End With

        _prefs.General.NumberOfItemsInGrids = CInt(txtNumberOfItemsInGrid.Value)

        With _prefs.DoNotShowMessages
            .HideNoResultsMessage = chkDoNotShowMessages.GetItemCheckState(0) = CheckState.Checked
            .HideCloseConfirmation = chkDoNotShowMessages.GetItemCheckState(1) = CheckState.Checked
        End With

        _prefs.Save()

    End Sub

Save That Email. As an Email.

Applications typically send a lot of emails.  At least, they should, since it’s a good, archive-able method for communication and confirmation.  Archive-able for the receiver, sure, but what about the sender – the application?  You have a couple of options: you can parse out all the fields of the email and stick them in a database or you can CC or BCC the email to a mailbox for archival.  What if you want to store the actual email that was sent?  You want the actual EML file.

Wouldn’t it be nice if the MailMessage object had a .SaveAs method?  It doesn’t.  Well then, wouldn’t it be nice if the SmtpClient object had a .SaveTo property?  It does, kind of.  And by using those properties, we can capture the actual EML file that is typically sent to a SMTP server for delivery.  Once we have that EML file data, we can save it to a file or to a database or wherever.

Quite simply, you need to set two properties on the SmtpClient object: DeliveryMethod and PickupDirectoryLocation.  This tells the SmtpClient to write the EML to a specified folder and the mail server will monitor that folder and pull it from there.  This is the code:

    Private Function GetEmailBytes(ByVal eml As Mail.MailMessage) As Byte()
        Dim smtp As Mail.SmtpClient
        Dim customFolderName As String
        Dim fileBytes() As Byte

        customFolderName = IO.Path.Combine(My.Computer.FileSystem.SpecialDirectories.Temp, Guid.NewGuid.ToString)

        IO.Directory.CreateDirectory(customFolderName)

        smtp = New Mail.SmtpClient
        With smtp
            .Host = "localhost"
            .DeliveryMethod = Mail.SmtpDeliveryMethod.SpecifiedPickupDirectory
            .PickupDirectoryLocation = customFolderName
            .Send(eml)
            .Dispose()
        End With

        fileBytes = IO.File.ReadAllBytes(New IO.DirectoryInfo(customFolderName).GetFiles.First.FullName)

        IO.Directory.Delete(customFolderName, True)

        Return fileBytes

    End Function

To explain the extra legwork involving directories, the SmtpClient writes the EML file with a GUID as a filename.  This prevents emails from overwriting each other.  In a multi-user environment though, how could we know which file was just written so we read the right file? So to be sure what we’re reading is our email, we create a unique folder to write the EML to and we know there will be only one file in there to read.

The GetEmailBytes method just returns the bytes of an EML file.  That’s the most flexible way to work with the data.  If you want to save that to another place with another name, just use IO.File.WriteAllBytes, like so:

    Private Sub SaveMessage()
        Dim eml As Mail.MailMessage
        Dim bytes() As Byte

        eml = New Mail.MailMessage
        With eml
            .To.Add(New Mail.MailAddress("anyone@home.com"))
            .From = New Mail.MailAddress("nobody@home.com")
            .Subject = "test save"
            .Body = "this is the body"
        End With

        bytes = GetEmailBytes(eml)

        IO.File.WriteAllBytes(IO.Path.Combine(My.Computer.FileSystem.SpecialDirectories.Desktop, "output.eml"), bytes)

        eml.Dispose()

    End Sub

Finding Solutions In Problems

I’m always looking for novel ways to accomplish something, even it’s totally inefficient… This sounds familiar.

Interestingly, the solution I was experimenting with in the last post is closely related to a problem I took up for resolution today.  In the inefficiency of the code, I found a very clever and unique way of splitting a string on a delimiter like a comma.  Now in this particular case, there was a much more efficient way to accomplish the end result, but I can appreciate the creativity in this query.

This is the code that was focused on:

declare @SortExpression VARCHAR(100) = 'state,zip',@SortOrder VARCHAR(10) = 'ASC'

SELECT STUFF(
(SELECT ', ' + Parsed 
FROM (
    SELECT SUBSTRING(@SortExpression + ',', Number,
        CHARINDEX(',', @SortExpression + ',',Number) - Number) + ' ' + @SortOrder AS Parsed
    FROM Common.dbo.Numbers
    WHERE Number <= LEN(@SortExpression)
    AND SUBSTRING(',' + @SortExpression, Number, 1) = ','
    ) t2
FOR XML PATH(''),TYPE).value('.[1]', 'varchar(MAX)'),1,2,'')

Whenever you see three SELECT keywords and XML together to result in a single value, something seems out of place.  This code takes two strings, “state,zip” and “ASC” and turns it into a single string “state ASC, zip ASC”.  That’s a lot of code for just that.

The interesting part of that query is in the middle, where the query accesses the a database with a table called Numbers which just has a list of numbers from 1 to 10000.  In another post, I created a list of numbers, the exact same thing, in memory using UNPIVOT and a CTE.  I could’ve done the exact same thing to fix this, but the eventual solution I came up with does the same with no recordsets at all to deal with.

Drilling into the inner query,

SELECT SUBSTRING(@SortExpression + ',', Number,
    CHARINDEX(',', @SortExpression + ',',Number) - Number) + ' ' + @SortOrder AS Parsed
FROM Common.dbo.Numbers
WHERE Number <= LEN(@SortExpression)
AND SUBSTRING(',' + @SortExpression, Number, 1) = ','

the thing I find so clever about this query, despite its inefficiency, is the approach it takes.  It uses a sliding window of characters, breaking on the comma, and looking for a trailing comma (that was manually added as a terminator).  When you see the results without the SUBSTRING criteria in the WHERE clause, it looks like:

Parsed

state ASC

state ASC

tate ASC

ate ASC

te ASC

e ASC

ASC

zip ASC

ip ASC

p ASC

And with the SUBSTRING check, the only rows returned are the ones that have a trailing comma.  Code like this makes your head hurt.  In fact, it’s so complex, you almost have to accept it at face value, thinking it’s just so complex, you’d better not touch it.  That is when you isolate and experiment, get the same results, prove it out with different values, and improve it.

So what’s the best replacement I could come up with?

select replace(@SortExpression,',',' ' +@Sortorder + ',') +
case when @SortExpression<>'' then ' ' + @Sortorder else '' end

One complete statement.  Whee!

Finding Solutions Before Problems

I’m always looking for novel ways to accomplish something.  Even if it’s totally inefficient, it still is a good exercise in problem-solving.  Sometimes you can find a good challenge by taking something simple, then making it complex.  For example, there is a simple way to get the first missing value within a non-contiguous range.

declare @i int=1
create table #t(value int)

-- Add numbers in multiples of 4
while @i<100
begin
    if (@i % 4)=0 
    insert #t(value)values(@i)
    set @i=@i+1
end

-- Get first available value
select top 1 t1.value+1
from #t t1
where not exists (
    select 1 
    from #t t2 
    where t2.value=t1.value+1
    )
order by 1

drop table #t

You can get all the next available numbers by removing the “top 1”, but what if the gap was more than 1 value wide?  You’d have to get the first missing values, then fill them or track them in a temp table/variable, then call the statement again.  In this example,  we do have gaps larger than one value.  We want to return 1,2,3,5,6,7,9,10,11, etc. This means we need to check for every value that is not in the #t table.  How would you be able to do that in a single statement?  In pseudo-code, you want to do something like:

select x where not exists (select 1 from #t)

but you have to have a FROM clause.  And that means you have to have the potential values somewhere to pull from.  You could make a temp table/variable and populate it with a range of values you want to compare against, but again, we want this to be done in one statement.

To solve this, you can make a small lookup table using UNPIVOT

select v
from (
    select 0 v0,1 v1,2 v2,3 v3,4 v4,5 v5,6 v6,7 v7, 8 v8, 9 v9
    ) n
unpivot(v for value in (v0,v1,v2,v3,v4,v5,v6,v7,v8,v9)) n

This gets you a resultset of 0 through 9.  Not a lot of values to compare against.  But structuring it as a CTE, you can make this 0-99.  You can join it again to get 0-999, and keep joining until you get the maximum values you need.

with Numbers(num) as
    (
    select v
    from (
        select 0 v0,1 v1,2 v2,3 v3,4 v4,5 v5,6 v6,7 v7, 8 v8, 9 v9
        ) n
    unpivot(v for value in (v0,v1,v2,v3,v4,v5,v6,v7,v8,v9)) n
    ) 
select (N1.num*10)+N2.num 
from Numbers N1
full join Numbers N2 on 1=1

Now that we have a table of lookup values, it’s trivial to find the missing values from our original table.

with Numbers(num) as
    (
    select v
    from (
        select 0 v0,1 v1,2 v2,3 v3,4 v4,5 v5,6 v6,7 v7, 8 v8, 9 v9
        ) n
    unpivot(v for value in (v0,v1,v2,v3,v4,v5,v6,v7,v8,v9)) n
    ) 
select lookupNum
from (
    select (N1.num*10)+N2.num lookupNum
    from Numbers N1
    full join Numbers N2 on 1=1
    ) n
where not exists(    
    select 1
    from #t t
    where t.value=n.lookupnum
    )
order by 1

One complete statement.  Whee!

A Toolbar of Your Favorite Menu Items? Interesting…

Originally posted at SOAPitStop.com – Sept 2, 2009

Remember that little idea that Office had a while ago that would hide infrequently-used menu items?  Wasn’t that a great idea?  For me, it was the very first thing I turned off after installing Office.  But I do understand what they were going after.  When applications do so much, every user is probably just using a subset of the whole application’s features.

The application that I’m writing is kind of getting like that.  A few versions ago, I created a toolbar on the side that was planned to be context-sensitive, so it would show actions based on what data was shown and available – kind of how Microsoft is now doing with the task pane.  Eventually, I may create or convert the toolbar to a task pane.  But as the application was growing, I had the same thought the Office designers had: each user probably only cares about 5 or 6 menu items at a time and those items should be as readily available as possible.  So instead of making personalized menus, I decided to create a Favorites toolbar.  This is similar to Microsoft programs where you can add toolbars and put menu items on them.

Because the application is in flux and because I am lazy, I didn’t want to go through the effort of creating a “Customize Toolbar” dialog.  I also didn’t want to have an extra dialog for “Add To Favorites”.  So what I did was allow menu items to be dragged onto the toolbar.  The proof-of-concept started as most do, just to see how it would work.  I got it going in under 150 lines of code, even less considering whitespace and definitions and all.

To quickly summarize the technique, I started by putting a toolbar container on the form, adding a toolstrip to hold the favorites, and adding a menu to hold the draggable items.

Then I added the code to allow the dragging of the menu items:

    Private Sub Menu_MouseMove(ByVal sender As Object, ByVal e As MouseEventArgs) _
        Handles mnuFirst.MouseMove, mnuSecond.MouseMove, mnuThird.MouseMove, mnuFourth.MouseMove, _
        mnu2ndLevel1.MouseMove, mnu2ndLevel2.MouseMove, mnu2ndLevel3.MouseMove

        Dim item As ToolStripMenuItem

        If e.Button = Windows.Forms.MouseButtons.Left Then
            item = CType(sender, ToolStripMenuItem)
            item.DoDragDrop(item, DragDropEffects.Copy)
        End If

    End Sub

Then the code to drop the items (the toolstrip needs to have AllowDrop set to True):

    Private Sub toolFavorites_DragEnter(ByVal sender As Object, ByVal e As DragEventArgs) _
        Handles toolFavorites.DragEnter

        If e.AllowedEffect = DragDropEffects.Copy AndAlso e.Data.GetDataPresent(GetType(ToolStripItem)) Then
            e.Effect = DragDropEffects.Copy
        End If

    End Sub

    Private Sub toolFavorites_DragDrop(ByVal sender As Object, ByVal e As DragEventArgs) _
        Handles toolFavorites.DragDrop

        Dim droppedItem As ToolStripItem

        droppedItem = CType(e.Data.GetData(GetType(ToolStripItem)), ToolStripItem)
        AddToFavorites(droppedItem)

    End Sub

    Private Sub AddToFavorites(ByVal item As ToolStripItem)
        Dim newItem As ToolStripButton

        newItem = New ToolStripButton(item.Text, item.Image)
        newItem.Tag = item
        AddHandler newItem.MouseDown, AddressOf FavoritesContext
        AddHandler newItem.Click, AddressOf FavoritesClick
        AddHandler item.EnabledChanged, AddressOf OnMenuEnabledChanged

        toolFavorites.Items.Add(newItem)

    End Sub

Then the code to route the click of the favorites to the real menu item

    Private Sub FavoritesClick(ByVal s As Object, ByVal e As EventArgs)
        Dim item As ToolStripItem

        item = CType(CType(s, ToolStripItem).Tag, ToolStripMenuItem)
        item.PerformClick()

    End Sub

That was really it.  Of course, then I had to persist the favorites in My.Settings and provide a way of removing the favorite menu item, resulting in the above-referenced AddHandler statement for FavoritesContext and a couple other methods for running through the menu items on the form load and close.  Then we need to disable the favorite button when the linked menu item is disabled, leading to the AddHandler for OnMenuEnabledChanged.  It just keeps growing.

Dating Advice

No, I certainly can’t help you get dates or have successful dates.  But I can offer a couple of functions that might help you work with dates in your code.  I suck at dates (all types).  Every time I want to calculate dates, I need to print a calendar from Outlook and count the days.  It’s pretty ironic that one of my previous projects was all about schedules.  And one of my current projects deals with delivery schedules as well.  So I can’t escape it.

I always would get frustrated because every month was different.  Every month started on a different day and had a different number of days.  It felt impossible to get all the different combinations.  But recently, I had a moment of clarity and realized some basic facts about a month.  Things like:

  • No month has less than 28 days
  • This guarantees every month will have 4 weeks
  • This guarantees there are no less than 4 and no more than 5 of every weekday in a month
  • The only weekdays that will have 5 occurrences will be the days in excess of 28.  These days can be accounted for at the beginning or end of the month – it doesn’t matter.
  • By extension, there are a minimum of 20 workdays in a month (Mon-Fri)
  • And, any additional workdays will be those in excess of 28 that are between Monday and Friday

Earlier attempts to figure out the number of workdays in a month resulted in a brute force loop that would run through every day from 1 to 31 and if the DayOfWeek was Mon-Fri, increment a counter.  Now, with these new guidelines, I can start at 20 and only deal with 0-3 excess days.  Like with this function:

Shared Function WorkdaysInMonth(ByVal d As Date) As Integer
    Dim daysInMonth As Integer
    Dim extraWeekDays As Integer

    daysInMonth = New Date(d.Year, d.Month, 1).AddMonths(1).AddDays(-1).Day

    For i As Integer = 1 To daysInMonth - 28
        Select Case New Date(d.Year, d.Month, i).DayOfWeek
            Case DayOfWeek.Monday, DayOfWeek.Tuesday, _
                DayOfWeek.Wednesday, DayOfWeek.Thursday, DayOfWeek.Friday

                extraWeekDays += 1

        End Select

    Next

    Return 20 + extraWeekDays

End Function

Using the other rules for weekdays, we know we only need to deal with the exceptions, to find out whether there are 5 weekdays in a month:

Private Shared Function NumberOfWeekdaysInMonth(ByVal weekday As DayOfWeek, ByVal referenceDate As Date) As Integer
    Dim firstDay As DayOfWeek
    Dim lastDay As DayOfWeek

    firstDay = New Date(referenceDate.Year, referenceDate.Month, 1).DayOfWeek
    lastDay = New Date(referenceDate.Year, referenceDate.Month, 1).AddMonths(1).AddDays(-1).DayOfWeek

    If New Date(referenceDate.Year, referenceDate.Month, 1).AddMonths(1).AddDays(-1).Day = 28 Then
        Return 4

    ElseIf lastDay >= firstDay AndAlso weekday >= firstDay AndAlso weekday <= lastDay Then
        Return 5

    ElseIf lastDay < firstDay AndAlso weekday >= firstDay - 7 AndAlso weekday <= lastDay Then
        Return 5

    ElseIf lastDay < firstDay AndAlso weekday >= firstDay AndAlso weekday <= lastDay + 7 Then
        Return 5

    Else
        Return 4

    End If

End Function

That one got a bit hairy because is the extra days started at the end of the week with a high DayOfWeek value, and ended early in the week with a low DayOfWeek value, we had to compensate at each end.  That’s the reasons for all the different IF conditions.  There’s also a specific condition for February’s 28 days.

Finally, a function to determine the first, second, third, fourth, or fifth weekday in a month.  Useful when calculating holidays like Labor Day and Thanksgiving.

Private Shared Function NthDayOfMonth(ByVal index As Integer, ByVal weekDay As DayOfWeek, _
    ByVal referenceDate As Date) As Date

    Dim firstDay As DayOfWeek
    Dim dayOfMonth As Integer

    firstDay = New Date(referenceDate.Year, referenceDate.Month, 1).DayOfWeek
    dayOfMonth = (7 * (index - 1)) + _
        CInt(IIf(firstDay > weekDay, 7 + weekDay - firstDay, weekDay - firstDay))

    Return New Date(referenceDate.Year, referenceDate.Month, dayOfMonth + 1)

End Function