Scott’s Weekly Source Code just looked at handling single instance applications using Visual Basic’s built in support. Rather handy, and desperately easy.
But it’s not how the shell does it.
Windows already has a defined way to handle this. Historically, this was DDE, which involved broadcasting Windows messages to initiate a DDE conversation. Naturally not the best performing option (ever wonder why the IE options are so slow at closing? They broadcast the options change notification to all windows). Command line args are limited in length, and you've got the added cost of creating a new process for each new file to be passed through.
So, in Windows XP and later, Microsoft added support for registering your application as a COM IDropTarget for the file type. When the shell wants to send you some files, it creates an instance of the COM object and passes it an IDataObject that represents the shell items for the files. In fact, because it’s giving you shell items, you’re not limited to files, but can now accept anything that can be represented in the shell’s namespace – printers, files within zips, any other virtual item. Try that with your command line args.
Implementing it is, well, what you’d expect from COM.
You can, if you’re feeling a bit daft, implement an in-process COM object, get your dll loaded into explorer and perform your own inter process communication with your app. Kinda defeats the point, though.
Or, more appropriately, you can register your application as the out of process COM server for the object. And then you need to implement the standard COM goo of registering a class factory with CoRegisterObject, and dishing out your IDropTarget when asked for it.
The shell calls your IDropTarget and passes in the parameters. So it all happens in your first instance.
And there's the subtle difference - it's less about single instance applications, and more about passing data through to the currently running app. Almost the same thing, but not quite.
Now, this method isn't perfect - it's no good if you just double click the executable again, or call it on the command line. And it's not terribly useful if you want to pass arbitrary data through (a command line arg saying "stop").
That said, there's nothing stopping your second instance passing through a custom IDataObject data format to the existing IDropTarget...
So you can get single instance working with this method, and getting shell multi-select verb support for free can be useful, too.
As ever, MSDN has the skinny.
Now, if only there were some C# code. After all, who’d want to use a Visual Basic dll that already solved the problem?