Timezone test

by Matt 27. October 2006 10:38

Just a quick post to test timezones. I'm posting this at 4.38pm GMT+1, 1538 UTC. I think there's a bug in SUB.

This isn't the post you're looking for...


Extended verbs in shell context menus

by Matt 27. October 2006 10:05

 A little while back, Tim Sneath posted a rather nice Vista tip.

Here's a trick that I don't think many people are aware of. Right-click on any folder on your Windows Vista machine while holding down the shift key. You'll see an extra context-sensitive menu item there: Open Command Prompt here. Just click on this menu and a command window will open with the current working directory set to the folder's actual location.

Now, this is a really nice thing. I've always installed the "command prompt here" power toy - now I don't have to. It's even better than the old version, too. It can automatically map a drive letter when you're browsing a UNC path, and it's available when right clicking on the background of a folder view, so you don't have to have the folder tree view visible.

And that's where you spot the bug. At least on my RC2 (5744) build, when you shift-right-click the folders in the folder tree view, it doesn't show up. This made me to wonder if it was a folder view thing. In other words, it wasn't a shell extension defined in the registry, but was a feature of the context menu in the folder view, something the tree view didn't have access to.

Fire up regedit (yes, UAC, I really meant to) and navigate to HKCR\Directory. There's a "cmd" key under both Background\shell and just shell. These both have a "command" key that specifies the cmd.exe command line to run. So, it's not a folder view thing, but a bug in the tree view. Fair enough.

The interesting part is in the "cmd" key. There's a value called "Extended". Knowing my shell extension terminology, there's a flag that gets passed through to a context menu extension called CMF_EXTENDEDVERBS. This tells the context menu to display an extended list of items - items that would normally be hidden. It's usually passed when you hold down shift. Looks like you can also specify it in the registry, too.

And this isn't just a Vista thing, either. It all works under XP, too. Handy thing to know.

Source: Tim Sneath : Windows Vista Secret #1: Open Command Prompt Here


Vista | Shell Extensions | Context Menu Handlers

Vista problems

by Matt 27. October 2006 02:35

I should have mentioned this in my last post, but I got all carried away about how nice it is.

It's not all a bed of roses. You can still tell at points that it's a beta product. Or at least, I hope that's why it's not working properly.

My biggest problem is power management. I can't let the computer go into sleep mode, because it never really recovers. The lights go and the disk flashes, but there's no screen. I guess I need newer display drivers (that I can't yet find on the interweb). Battery life doesn't seem brilliant, either - I guess the graphics keeps things busy.

The only other real problem has been trying to get Visual Studio Team Explorer installed with User Access Control. It just wouldn't do it. I had to disable UAC, reboot and try again. Unfortunately, when I was trying again, I ran a different setup program that worked first time, so I don't know if it was a compatibilty problem with the setup wrapper, or UAC (which is back on again - it's actually quite a nice security blanket).

If those are my only problems, I'm sorted.



Me and Vista, sitting in a tree...

by Matt 26. October 2006 10:32

Yeah baby. It's Vista time.

Part of a new role I'm in at work has me looking at .net 3. I've held off this for a while - not enough time to mess about with the changing targets of monthly CTP releases. But seeing as we're now very close to Vista shipping (and with it .net 3) and 'cos it's now my job, I've got myself a nice shiny new laptop, and a nice shiny new OS.

What can I say that's not been said by so many other blogs and reviews?

It looks nice - I like the glass and the fading, and the cute little animations. It's all good fun to show off.

I still haven't found a good use for the sidebar. The analog clock is nice, but I don't think it's compelling enough to keep it running.

User Account Control - the contentious one. To be honest, it's not that bad at all. It's slightly annoying at the moment, but that's because I'm installing lots of stuff, and so seeing it a lot. I expect this will settle down under normal usage. It's a bit jarring though - the whole screen disappears for a second while it switches to the secure desktop. A nice transition would just make the whole experience a lot more pleasant. The one thing I don't like is trying to do something in Explorer that requires permissions. Explorer prompts me to see if I want to get prompted by UAC - so I get two prompts to click through.

And speaking of Explorer - it's all change there. I really like the new folder view. I like being able to click on the headers to sort, group, stack and filter. I like the new views and the resizing thumbnails. I like the breadcrumb bar. I like the favourite links. I don't like the little folder tree - it's too small and cramped and it makes the interface busier and messier, and keeps coming back when I close it. But on the plus side, I'm finding I'm using it less and less as a means of navigating. I'm beginning to care less about where things are stored on the disk. UAC stops me messing about in Program Files, so I don't go there as often. Everything else, I just search for.

Oh yes, search. I'm already a fan of Windows Desktop Search, but this just blows it out of the water. I don't know if it's because I've got a faster machine with more memory, but it's not just faster, it's instantaneous! Perhaps this is an indication of the performance improvements in Vista - which just seems rock steady and dead quick - I've been trying to batter it with VPC's and Visual Studio and it's taken everything I can give it without breaking a sweat. Search in folders is brilliant too - the word wheel effect in Control Panel is great. No more hunting! And I love searching from the start menu. No more navigating to find something, just windows key and type. I really miss that when I'm back on XP. But I don't like the size of the start menu! I think getting rid of the flyout menus for All Programs was a good idea, but now we're stuck with the list in a tiny little pane inside the start menu. It's not very good for browsing. Searching, yes. Browsing, no.

But perhaps the best thing is the quality of text. Again, I don't know if it's the new machine, but the picture is fantastically crisp. It's a huge difference between IE or Visual Studio on XP.

So, yes, Vista is lovely. It's not as minimally chic as the Mac, but it's got it's charm, and it's about time we had a new Windows to play with.



Registering your protocol handler. Actual code.

by Matt 18. October 2006 15:32

Enough procrastination. I've set the lofty goal. I've worried about documentation. I've fretted about SDKs. I've even lectured on how I want my project set up. I haven't written a single line of code. For shame.

To be fair, this is going to be a little dull, so let's rush through the boring bits. Following my own advice, I've created an ATL project called msfeedph and deleted all the unnecessary bits. I've also added the include directories of the WDS SDK, the msfeeds API mini-SDK + the SharePoint Portal 2001 SDK to the project. I've then included wdsSetup.h in the stdafx.h.

Now we're ready to register our protocol handler. The docs say that we need to call ISearchManager::AddProtocol. Looking at wdsSetup.idl, we can see that the SearchManager object implements ISearchManager, so let's create one of those, and call AddProtocol. We need to pass in a protocol name, and the ProgId of an object that will handle that protocol. We'll use "msfeed" as the protocol name, and "Sticklebackplastic.MsFeedProtocolHandler.1" as the ProgId. We'll also do a reciprocal RemoveProtocol. Here's the code:

STDAPI DllRegisterServer(void)
    HRESULT hr = _AtlModule.DllRegisterServer(FALSE);
    CComPtr<ISearchManager> spSearchManager;
    hr = spSearchManager.CoCreateInstance(CLSID_SearchManager);
        hr = spSearchManager->AddProtocol(L"msfeed", L"Sticklebackplastic.MsFeedProtocolHandler.1");

    return hr;

STDAPI DllUnregisterServer(void)
    HRESULT hr = _AtlModule.DllUnregisterServer(FALSE);
    CComPtr<ISearchManager> spSearchManager;
    hr = spSearchManager.CoCreateInstance(CLSID_SearchManager);
        hr = spSearchManager->RemoveProtocol(L"msfeed");

    return hr;

Excellent. Just one small problem - it doesn't compile. CLSID_SearchManager is an undefined symbol. We need to add a new file, that I've called DefineGuids.cpp:

#include "stdafx.h"
#define INITGUID
#include <wdsSetup_i.c>

This just includes the C part of the file generated from wdsSetup.idl, which fortunately for us, declares the GUIDs from the idl file.

Now we compile and register and all is well. We don't yet have a "Sticklebackplastic.MsFeedProtocol.Handler.1" class so nothing appears in WDS yet, but it's a start. We can check the registry (HKCU\Software\Microsoft\RSSearch\ProtocolHandlers) and see that it's there.

The next step is to set the default urls that WDS will crawl.

You can download the current code here.


Windows Desktop Search

The continuing saga of StaticFileHandler - meet DefaultHttpHandler

by Matt 9. October 2006 16:32

I bloody well love Reflector. It's just magic.

After discovering that StaticFileHandler doesn't handle conditional gets and ranges, I've dug a little deeper, and, unsurprisingly, the plot thickens.

In .net 1.x, you could use StaticFileHandler to serve static files from an asp.net application - just use the IIS wildcard mapping so that all files went through the asp.net ISAPI dll, and modify your web.config.

In .net 2, things are a little better. Now, the global web.config file catches any GET, HEAD or POST requests to any file that hasn't already been handled and passes them off to DefaultHttpHandler, not StaticFileHandler. (Any other verbs get passed to HttpMethodNotAllowedHandler. That one's internal. 3 guesses what it does?)

This is quite a nice class, at least in theory. It's an asynchronous http handler that will serve static files. The bad news is that it serves static files using StaticFileHandler, with all the issues that has (no support for conditional gets or ranges). But the good news is that if you haven't modified any headers (such as caching) and you haven't already written something to the response stream, and the response stream doesn't have a filter (such as compression) and you're running in process in IIS6 then the request is passed back to IIS for processing. IIS will do the decent thing, and serve the static file with support for conditional gets and ranges and all will be good. If any of that exhaustive list is not true, then StaticFileHandler gets to serve the content (boo, hiss!). Note that you can still get compression, but you'll have to configure it at the IIS level now, not the HttpModule level. That's the good news.

This might seem of limited use - if you can't modify the headers, or add a compression filter to the output stream, what's the point? Well, you can monitor and log the request, for one thing. And more importantly, you can derive from the public DefaultHttpHandler and override the virtual OverrideExecuteUrlPath to route you off to somewhere else and IIS will serve it (sounds just like I was trying to do originally!)

Unfortunately, this class has it's problems, too. It doesn't allow you to route to your own static file handler. Even worse, though, is that you can only specify an override of the path when it decides (via conditions outside of your derived class's control) that IIS should serve the file. If it decides that StaticFileHandler should serve it, you have to have implicit knowledge of the inner workings of the DefaultHttpHandler class and know to do a Context.RewritePath in OnExecuteUrlPreconditionFailure. This is mentioned (albeit obliquely) in the docs for OverrideExecuteUrlPath. And I can't roll my own version of this as it uses internal methods to pass control back to IIS.

As long as you're strict about your global setup, this might be useful. I'll have to do a test to see if it'll fit my requirements, but this feels a little fragile - I really don't want it falling back to StaticFileHandler. I think I'll follow up on the Virtual Path Provider idea, or just buckle down and write a BetterStaticFileHandler.



Serving files from App_Data #2 - the lazy approach

by Matt 6. October 2006 17:18

I prefer "conservation of energy" myself.

Last time, we looked at why I would want to do such a crazy thing as serving files from the App_Data directory, when asp.net already forbids it for security reasons. We even looked at a rather naive approach for doing it, and saw that it was missing some rather important features - namely efficient memory usage, mime types and conditional gets.

This time, we'll get someone else to do all the heavy lifting for us. We're after a http handler that can serve static files. Asp.net has StaticFileHandler. Isn't that handy?

StaticFileHandler is how you can serve html, css, gif and jpg files when you route all files through asp.net. It pretty much does everything we want it to - it uses HttpResponse.WriteFile for small files and HttpResponse.TransmitFile for large files. It uses the undocumented and internal System.Web.MimeMapping class to map between an extension and it's mime type (hard coded, surprisingly enough) and most of all, it returns etags. It even ensures the file exists and you've got permissions and has code to handle resuming broken downloads. We've hit the jackpot. Almost.

So how can we use this? I was going to present a code sample here, but that "almost" above is stopping me. This isn't the post I set out to write - I can't advocate this method (which is a shame, 'cos I liked it - it was simple, looked like it worked, and was only slightly hacky). But i'm going to tell you about it anyway.

I created a class I called RewritingStaticFileHandler. It had a static constructor, which created an instance of StaticFileHandler and stashed it away (bizarrely, StaticFileHandler is internal, even though it's used in the default machine.config. Since it's in that file, I reckon it's fair game, so spin one up by reflection. This should have been a warning sign that things weren't going to work out too clever).

The ProcessRequest method of my new http handler called HttpContext.RewritePath, and passed in a new path in the App_Data folder with the file part coming from a query string parameter (danger Will Robinson!). It then called ProcessRequest on the stashed StaticFileHandler and voila - we're now serving files from App_Data, returning an etag, populating the mime data and I've only written 5 or 6 lines. Lovely. Almost.

Anyone spot the security hole? I'm setting the rewrite path to "~/App_Data/Images/" + Context.Request.QueryString["imageId"]. Pass in ../password.txt and it's going to serve the password.txt file from the App_Data directory. Not good.

This is quite straightforward to fix:

private void EnsureCanonicalPath(string path)
  string mappedPath = Context.Server.MapPath(path);
  if (Path.GetFullPath(mappedPath) != mappedPath ||
    throw new HttpException(404, "Not found.");

Turn the path into a physical path, call GetFullPath (which always returns a canonical path) and ensure it starts with the physical path of the images directory. This way you're ensuring that you only serve files from the images subdirectory.

So why am I not happy with this approach? Well, it's a little bit hacky using reflection, but that's not it. I fired up Reflector to check what StaticFileHandler did under the covers, and I don't like what I see. It has a number of bugs. Where do I start?

There is a method called SendEntireEntity, which returns a bool. This is the only time the incoming etag header is checked. If you're requesting a range, this method checks the etag header and if etags match, returns false - send a range. Otherwise it returns true. The return value is ignored and it always sends back the entire file.

Any conditional get headers, such as If-Unmodified-Since, Unless-Modified-Since, etc are not checked, so the etag is useless there.

The etag value itself is derived from the last modified time of the file concatenated with the current time. This is generated at the start of each request. So, if you are asking for a range, you'll pass in an etag you received when you started downloading the file. This etag will include the time of the request. The new request will get a new etag. So the etags are always different. Again, useless.

So why did it look like it was working? For small files, it sends back a cache expiry date for a day hence, so IE just wasn't bothering to request the file again. But when it will request the file, the handler will just return it in full.

I'm not very happy about this. I was expecting StaticFileHandler to be a bit better than this. Having this kind of problem has lots of implications - I'm going to have to make some changes at work based on this. Bah. Not the post I wanted to write. Still, it means there's another one to add to the queue - BetterStaticFileHandler.

And where does that leave serving files from App_Data? Well, this solution will still work, it's just not the most efficient - definitely not something I'd recommend in an enterprise, but probably not too much of a problem for a blog. It's also not the nicest solution, so let's have a look if there's anything more elegant. Let's have a look at Virtual Path Providers.



SDKs for search protocol handlers

by Matt 5. October 2006 13:36

Sheesh. After finally detailing how I setup a blank ATL COM project, I thought I might as well actually begin to develop something. Of course I'd forgotten something. We've already seen what a mess the documentation is in, but what about SDKs?

Firstly, we're going to need the Windows Desktop Search API SDK. This is a nice little zip file that includes a couple of idl files. There's a wdsQuery.idl that gives a coclass and an interface for issuing queries, a wdsView.idl that appears to allow you to host the WDS results viewer (with previewer?). The one we're interested in, though, is wdsSetup.idl, that gives a few interfaces to aid in registering a protocol handler. It even includes the CLSID of the object that implements ISearchManager that I was worrying about last time.

But it doesn't include the ISearchProtocol interface itself.

Looking at the Other Resources section of the MSN Search Toolbar guide (isn't it called Windows Desktop Search these days, guys?) there's a link to some documentation on protocol handlers (as I pointed out last time), and there's a link to a knowledge base article that includes a link to download the 2001 SharePoint Portal SDK, which includes the definition of ISearchProtocol. We're going to use this.

It's probably worth getting the latest Platform SDK, too. You could go cutting edge and get the latest beta of the Windows SDK - but bear in mind that it includes the beta bits of the .net 3 framework...

Of course, since the whole point of this protocol handler is to search IE7's RSS feed store, you'll need headers for that too. They are included in the Windows SDK, or you can get them on the cheap from the RSS team blog.

Right. I think we're set. But I've said that before.


Windows Desktop Search

Serving files from App_Data #1 - the naive approach.

by Matt 5. October 2006 03:52

Don't worry, I've not gone crazy.

Asp.net 2 doesn't let you serve files from the App_Data folder, and for good reason - security. It's a folder your web app has write access to (a standard place on all installations without having to get your ISP involved - halellujah!) so in goes your databases and various other data files. You don't want anyone to be able to just download these, right? Right. Well, most of the time, anyway.

If you're dealing with a site that has mostly user-generated content, you're going to have to write a lot of data. The place of least friction for this data is of course App_Data. Think of a blog site (cough, SingleUserBlog, cough); all of the blog posts are going to be stored in the App_Data folder, perhaps as flat files or a database. And this is fine, because you don't serve those files directly - they're content to go in a page. But what about uploads? How can I include images in my blog? Uploading them is no problem, just chuck them in App_Data. But I can't serve them.

This is exactly the kind of solution a custom http handler is intended to solve. And the naive approach is to simply call HttpResponse.WriteFile and congratulate myself on a job well done.

So what's wrong with this?

Firstly, WriteFile reads the whole file into memory before chucking it to the client. For large files, it can fail. This isn't the best. But it can be worked around. I can use HttpResponse.TransmitFile to stream direct to the client. Or I could easily roll your own (this Microsoft KB article shows how).

Secondly, I'd have to add my own logic to setup the mime types. Boring, but still not reason enough to call it naive.

The third reason is the biggie. It's the single biggest mistake you see with custom http handler implementations, and that's forgetting caching and conditional gets. Bandwidth isn't free, and yet most http handlers will just serve the file. Again and again and again. To do things properly, I'd need to create and check etag values, and handle the various combination of the http headers (If-Unmodified-Since, Unless-Modified-Since, etc).

I could make this point into a blog post by itself. Fortunately, Kent Sharkey has already done it. Go read. Please.

Wouldn't it be nice if I could get someone else to do all this heavy lifting for me? (And I still haven't forgotten security.)



The prototypical ATL COM project

by Matt 2. October 2006 16:14

Now there's a title to make your pulse race.

This post is ostensibly another in the how-to-write-a-Windows-Desktop-Search-protocol-handler series, but it's really just what-Matt-does-to-write-a-COM-object. I've done it for just about every COM object I've written; I have a feeling I'll be referring back to it.

And yes, that title does say ATL. It's C++ time.

I'm not going into the C++/.net COM object argument just now. I will at some point - I've got plenty to say about it - but for this, I'm using C++. Not everything is a .net shaped nail.

Of course, these tips make certain assumptions. I'm talking about COM objects that are in-process and that only implement other people's interfaces (think shell extensions, addins for Office, Visual Studio or Windows Desktop Search. Pretty much any kind of addins). Some of this won't work for situations such as scriptable components, custom objects for a custom application and so on. Your mileage may vary.

Right then. Fire up Visual Studio and create a new ATL project. We want a dll and we don't want it attributed. I'm a bit of a Luddite with regards to C++ attributes. I just know it's a code generation thing, and I'm old-skool enough to write this stuff long-hand, thanks. (That said, I'm reading the link above, and it does look rather interesting. Might have to have a play with this...)

I'm going to assume the project was called "dllname", and I'm going to leave it as an exercise to the reader to substitute "dllname" for the right name for the rest of this post.

1. Remove proxy stub project

I haven't once used this. I'm not implementing my own interfaces, so I don't need to provide any custom marshalling. Right click on the PS project and delete it.

2. Get rid of the typelib

Again, I'm not implementing my own interfaces, so I don't need to describe them to anyone. If I don't remove the typelib, ATL will register an essentially empty typelib in the registry for me, and will store the typelib as a resource in my dll. Both of these are unnecessary bloat. Trivial, yes, but easily prevented (we should still try to be as streamline as possible, even in this day of cheap memory and storage).

Firstly, open up the dllname.cpp file, and modify DllRegisterServer. Simply add a FALSE to the arguments, so that the typelib isn't registered:

HRESULT hr = _AtlModule.DllRegisterServer(FALSE);

Do a similar thing for DllUnregisterServer:

HRESULT hr = _AtlModule.DllUnregisterServer(FALSE);

Now we need to remove the typelib from the resources. Open dllname.rc, and in the resource view, right click on the dllname.rc node (not the dllname project node!). Select "Resource Includes". You're going to get a dialog with two big edit boxes. In the bottom one will be the line:

1 TYPELIB "dllname.tlb"

Delete this.

Don't delete the .idl file from the project! This does create the type library (.tlb) but the ATL wizards need this file.

3. Don't register an AppId

Again, never needed this. Open the dllname.rgs file and remove the AppId section. Easy peasy.

4. Enable QueryInterface debugging

This one's a corker. Saved the best til last. Once you've done this, every time a client calls one of your objects' QueryInterface methods, you'll get a string written via OutputDebugString with the class name, the name of the interface requested (from the registry) and a "failed" marker if the interface isn't supported.

This is an absolutely essential debugging aid when working with COM. Quite often the documentation will tell you what all the interfaces are, what each one does, but not which combination of interfaces to implement on a given object. Or, it only lists the interfaces for that particular API domain, and don't list the standard COM interfaces required, such as IPersistFile, etc.

Edit you stdafx.h and define _ATL_DEBUG_QI before including any atl header files.

Note that there's a bug in AtlDumpIID (the function that does all the magic) in atlbase.h. If the interface name isn't found in the registry, the code is supposed to output the raw IID, but the logic is wrong, and nothing gets output. It's fixed in Visual Studio 2005 and later, so if you have anything earlier, check out Craig Skibo's post on the problem. He offers a solution that you just have to copy into the file. It's a little verbose (I'd have just set a flag on success and checked that at the end of the file rather than have all those goto's...)

A top tip in conjunction with this is to use sysinternal's DebugView to capture all those messages to OutputDebugString, even when you're not in the debugger. Oh, and if you don't know what a raw IID actually refers to, don't forget Google. Or install the Platform SDK and Windows Desktop Search.

5. Cleanup object rgs scripts

Now, this one is for when you create a new object, rather than the first steps of a project like the rest. So, go on. Create an object. Now open it's .rgs file. The first thing we can get rid of is the typelib reference. We're not registering that typelib any more, so we can get rid of that. And I always get rid of "Programmable", and if you're not using ProgIds, get rid of those too.

And that's it. Just a couple of simple changes that I make every time I start a COM object.

PS. Whatever development you're doing (C++/.net/asp.net/whatever) you really, really want to use Microsoft's symbol server.


Windows Desktop Search


Month List


Comment RSS