xunitcontrib-resharper 1.0: Stick a fork in it - it’s done

by Matt 4. April 2013 04:27

It might have taken 4 years, but I think it’s finally time to declare the xUnit.net test runner for ReSharper ready for 1.0 status.

This is a big update, and a long post (but there’s screenshots!) so let’s cut straight to the chase. If you want a download link, go straight to CodePlex and grab it. If you’re interested in what’s new, let’s start with a bullet list:

Where’s the ReSharper 8.0 EAP support? The code’s there, it’s ready to go. But the current EAP builds have an issue that prevents the test runner being loaded unless it’s in the product install directory, so it can’t work for plugins right now. This should be fixed soon.

Right. Screenshot time.

PropertyData attribute support

The PropertyDataAttribute tells xunit to get data for a Theory parameterised test by calling the named property’s getter. It relies on a magic string. And we all know that magic strings break when you rename the thing its referring to. Wouldn’t it be nice if ReSharper could help here?

And of course, it can. This release tries to add a reference between the string value of the attribute declaration and the property it’s referring to. So, if ReSharper can’t find a property with the same name as the string, it’s displayed as an error:

Cannot resolve symbol for PropertyDataAttribute

If it can find it, then it happily takes part in the ReSharper world of loveliness – ctrl-click navigation, included in find usages, renamed during refactoring, and it even marks the property as in use.

Property data highlighting usages

And there’s code completion (oh look! That’s Visual Basic!):

Code completion with PropertyDataAttribute

And of course, it works with the overload that takes a type name as well as a property name:

Code completion on property data attribute with a different type

Traits (categories) support

NUnit and MSTest both have a Category attribute. This takes a single string value, and ReSharper can use this to group tests, so you could, for example, run all “logging” tests at once. The xunit approach is a bit different, though. Instead of a Category attribute, xunit has the more general Trait attribute, that takes two string parameters, a name and a value. It’s trivial to model category with this, just use a name of “category”, but what do you do with the other trait values?

This answer is of course obvious, especially in hindsight (this post on how the TFS team had implemented it in the Visual Studio test runner was the light bulb moment for me). If you have two strings, and need to make one, just concatenate them. So, name/value traits are formatted as “name[value]” and this is treated as the ReSharper category. The name “category” gets special treatment. As long as the name is “category”, regardless of case, only the value is used. This allows for usage such as [Trait(“category”, “logging”)] to be grouped under the “logging” category, and something like [Trait(“owner”, “matt”)] to be grouped under the “owner[matt]” category. Note that only “category” gets the special, case insensitive, treatment – “owner” is not the same as “Owner”.

Trait showing category

And there is code completion here too, on both the name and value (yes, more VB!):


RunWith support

This has been one of the longest outstanding issues. The RunWith attribute allows you to change how the tests for a particular class are run – you get to take over the test discovery process. The default process looks for all methods that are decorated with a Fact or Fact derived attribute, but you could create a test class command that returns tests that being with “Test” or “Should”. Or one that calls a method in the class to get the tests to execute, or which runs the tests in a specific order.

The tricky thing here is that the ReSharper plugin can only go on what’s in the source code – it can’t run arbitrary code to find out what methods (if any!) should be tests. And it might be a random process, that changes each time the code is run. Sounds a lot like Theory support, in fact.

When the plugin encounters a class decorated with the RunWith attribute, it marks the class as being a test class, but it doesn’t mark any methods as test methods, even if they’re decorated with the Fact attribute – this could be a red herring if the custom RunWith test discovery is based on method name.

When the class is run, the methods are dynamically added to the test runner results window. Double clicking on the test will navigate to the appropriate method, and individual methods can be run and debugged just as with normal methods and theories. (The PrioritizedFixture shown here comes from the xunit samples)


Live Templates

There have been live templates in the release for years, but they’ve been a bit neglected – they were still in the old ReSharper 4.1 xml format! I’ve given them a bit of love and updated and added to them, and they’re now a first class part of the plugin.

The best bit is that you don’t have to do anything to get them – the plugin adds them to the settings system automatically. Just go to the Templates Explorer and see all the new templates tagged “xunit” in the C# section.


Wherever possible, the shortcuts have been chosen to be mnemonics of the assert, “ae” is Assert.Equal, “at” Assert.True, “athr” Assert.Throws, “adnt” Assert.DoesNotThrow, and so on. There are a few more for creating test methods, including “fact” to create a method decorated with [Fact], “fa” expands to [Fact] (Fact Attribute), xtc expands to an Xunit Test Class, and so on. Check them out in the Templates Explorer.

If you don’t like the shortcuts, or the code that’s generated, simply edit them. The changes are saved in your settings by default, and don’t override the shipped templates. (Although if you’re editing them, why not send me a pull request with the changes?)

But if you really don’t like the templates, you can disable them completely, by going to the ReSharper –> Manage Options dialog and unchecking the “xunitcontrib templates” settings layer. This removes them from the system completely.

Manage settings layer showing custom templates layer

External Annotations

External annotations have been in the zip file for years, too, also neglected. ReSharper’s analysis engine is very clever, but it can be even smarter if you give it more context by annotating your code with certain attributes. You can tell ReSharper when a return value or parameter will be null or not null, or if the method will throw an exception if a certain parameter is null, not null, true or false. You can even say that a lambda is being used immediately, rather than stashed away for later use, which limits the scope of a closure, and stop ReSharper showing you the dreaded “access to modified closure” or “implicitly captured closure” warnings.

And if you don’t have access to the source to add these attributes? Then you create an external annotations file, which maps annotation attributes to type members. The annotations for xunit have been updated to xUnit 1.9.1, and are now automatically installed.

Unreachable code due to annotations

Aborting test runs

This was something I hadn’t noticed about the test runner. When the tests are running, hit the stop button. The button changes to a hand icon, but the tests keep running. Hitting the button a second time stops the tests immediately. What’s going on here?

The first hit signals to the test runner process that you want to abort, and waits for the test runner to comply. If it doesn’t stop in good time, and you hit it again, it forcibly kills the test runner process.

Turns out, the xunit runner wasn’t paying any attention to the signal to stop, so the only way to abort a test run was to kill the process. And then the shadow copy cache didn’t get deleted, leaving folders lying around in your temp directory.

This release now plays nicely with the stop button, and will stop after the current test is run. If you forcibly end the run, the cache gets cleaned up correctly.

Update notifications

And finally, because no plugin should be without them, the plugin will now check for updates, and let you know if one is available.

What’s next?

I need to test these changes in the standalone dotCover test runners, and get a 1.0 release out there. And then, it’s full steam onto version 2.0, which will be all about the xunit 2.0 rewrite.

Until then, go get the 1.0 version, and go write lots of lovely xunit tests.

Tags: , ,

New ReSharper plugin - bringing the preview tab back

by Matt 1. February 2013 03:12

ReSharper 7.0 brought full support for Visual Studio 2012. One of my favourite little features was preview tab integration.

The preview tab is reusable, rather than the normal, persistent editor tabs. It displays one file at a time, and the content is replaced when a new file is navigated to. This is a great way to reduce the number of open tabs in the editor, especially if you’re just browsing a codebase. If you want to edit a file, just start typing, and it seamlessly becomes a normal editor tab.

Also, it’s a nice purple colour.

The purpley goodness of the preview tab

ReSharper 7.1 changed the behaviour, using the preview tab much less, and now normal editor tabs abound and multiply.

I’ve taken this rather personally. It’s messed up my workflow. I’m back to the bad old days of VS 2010, all too often declaring Document Bankruptcy and closing all windows, even those I’m editing on.

Unsurprisingly, I don’t agree with the arguments for the change – primarily, that if you’re navigating directly to a type or symbol, it’s so you can edit the file.

That’s not how I work. I read code as much as I write code, and tend to have to navigate to the right place before I can change anything. That can open more files than I actually need to work. The preview tab is brilliant for this – I get to read the file and navigate away without it cluttering up the editor. If I want to edit it, I simply start typing. And if I need to get back to a file that’s been replaced? Use ReSharper’s Recent Files popup, and bob’s yer uncle (you don’t use ctrl+enter, do you?)

ReSharper's recent files popup window

Fortunately, as is the way with ReSharper, it’s extensible and can be (ahem) fixed. So, if you want to get back to purple preview tab perfection, here’s a nice little plugin that forces all navigation to go through the preview tab. Source and binaries are on GitHub, and you can find the download link on the Release Notes page.

(ReSharper 7.1’s use, or lack of use, of the preview tab isn’t arbitrary. Direct navigation opens a normal editor tab. Browsing from a tool window, by hitting enter, or ctrl+alt+up/down to browse previous/next occurrence uses the preview tab. I just happen to want direct navigation to use the preview tab, too)

Tags: , , ,

New release of the NuGet ReSharper plugin

by Matt 24. January 2013 04:23

It’s been a little while since I released the NuGet plugin for ReSharper, long enough for a new release of NuGet.

And of course, the plugin stopped working.

Oh, and if you’re using VSTS for Database Professionals, and it seems to be crashing a lot, that’s my fault, too.

Fortunately, I’ve just pushed a new version, 1.1, that fixes these issues. The download link is on the Release Notes GitHub page.

What went wrong?

I did, frankly. I passed a package folder path instead of a repository package path, and it just happened to work in NuGet 2.1. A change in the implementation details in NuGet 2.2 and it was now using the wrong path and couldn’t find the package to install. Garbage in, garbage out. This meant the reference wasn’t added, or a file reference was added and dependencies weren’t, usually silently, so you didn’t know something had gone wrong. The fix was quick, and I added better error handling and reporting, so if something goes wrong, it will now fail more visibly, and without falling back to adding file references.

The crashing VSTS DB Pro issue was much more interesting, because it’s seemingly unrelated to NuGet, ReSharper or the plugin. Three virtual machines later, I could recreate it, by using Windows 7 and not installing VS 2012. Or, more accurately, by ensuring that .net 4.5 wasn’t installed. I had compiled the plugin against .net 4.5, and when running with just .net 4.0 installed, it would cause VSTS for DB Pro to crash. I still have no idea why – there were no useful exception messages or any other clues. A quick recompile later, and no more crashes.

One little enhancement made it in – update checking (inspired by Julien’s ForTea T4 plugin). ReSharper will notify you when a new version is released.

So, apologies for the nasty bugs – get the latest version from the link on the Release Notes.

Tags: , ,

xunitcontrib-resharper 0.7 - ReSharper 7.1 and stuff

by Matt 20. November 2012 03:39

ReSharper has just released 7.1, so I think it’s only fair to update the xUnit.net test runner to support it.

And it would be mean not to throw in a couple of bug fixes while I’m at it.

Firstly, there’s a fix to support filtering out test usages when using ReSharper’s Find Usages command. Frankly, I thought this was working ages ago, but it looks like I broke it some time back. Sorry folks.

When in the Find Results window, displaying the usages of a type, method, property, etc. you can filter out different types of usage – read/write, invocation, usage in attributes or documentation, and, what’s interesting here, usages in tests. This means you can hide any usage of a type from a test method or class. Simply toggle the “Show Unit Test Usages” menu item in the filter, and the Results window will hide and show the test usages (note that the test usages are using the test icon that appears next to the test method in the code editor).


The second bug fix relates to Theory data rows. xUnit.net uses class and method names as a means of identifying a test. This is guaranteed to be a unique value, as the test method doesn’t run more than once. For Theory parameterised tests, xunit uses the class and method name, but also uses a display name, which includes the parameter values, such as “ShouldBeUppercase(s: SAUSAGES)”.

There is a problem here. If you have two rows of data with the same value (“SAUSAGES”), the test runner treats these as the same test. This is a case of “doctor, doctor, it hurts when I do this”. Don’t do this. You’re passing in the same data, you’ll get the same results, it doesn’t matter if it looks like there’s only one test.

Or so I thought.

What happens if the data being passed in is an array of strings? xunit will call ToString on the parameter value, and come up with a display name like “ShouldBeUppercase(s: System.String[])”. Clearly, there are going to be collisions here. So now, the test runner keeps a track of names of tests it’s already run. If it tries to run the same test again, it changes the name, appending a call number, e.g. “ShouldBeUppercase(s: System.String[])” and “ShouldBeUppercase(s: System.String[]) [2]”, “… [3]” and so on.


And that’s it for this release. I smell a 1.0 version coming, finally. There are a couple of big ticket changes I want to make first. I want better support for PropertyDataAttribute (IntelliSense, find usages, ctrl-click navigation, etc) and InlineDataAttribute, and there’s a good memory optimisation I need to make for editing a large test file. If there’s anything else missing or broken, now’s a good time to let me know!

As ever, download it from Codeplex, install it with the handy batch file, let me know any issues.

Tags: , ,


xunitcontrib-resharper 0.6.1 - mostly bug fixes

by Matt 2. August 2012 02:42

Seems like this blog is becoming just an announcement service for new versions of my xUnit.net test runner for ReSharper. Bit of a maintenance release, this one. Here’s what it’s in:

RTM support for ReSharper 7.0 RTM:

The 0.6 release worked just fine with the RTM, but here’s the official build.

Bug fixes:

  1. The wrong dll was used when you switched between Debug and Release. Yikes!
  2. Adding a method to the end of a class, and then trying to run all tests in a class could cause the whole class to fail, as it tried to run a method called “???”
  3. If you used non-public test methods, you couldn’t double-click to navigate in the test sessions or test explorer views
  4. Exceptions were thrown when trying to parse a source file that contained static public properties and had a build action set to None. Rather a specific one, that.

The most interesting change is for placing tests in nested classes. When displaying a test method in the UI, it’s usually displayed as the class’s short name, e.g. For “Name.Space.ClassName”, you see “ClassName”. This scheme fails when you have nested classes, and multiple nested classes share a name, but have different parent classes. Like this:

public class AddressTests
  public class ToStringTests
    [Fact] public void ShouldReturnFormattedAddress() { /* ... */ }

public class PersonTests
  public class ToStringTests
    [Fact] public void ShouldReturnForenameAndSurname() { /* ... */ }

(Check out Phil Haack’s post that nicely describes why you might want to do this). The problem is that the test runner would display this:


making it very hard to distinguish which test was which. Now they display like this:


In other words, nested classes include the name of their parent class in the UI. I’m not exactly enamoured of the naming format; using a “+” to separate the class names isn’t very well known outside of Relection, but at least it gives you the chance to know which test is which. It might change in a future version - there’s an outstanding bug for ReSharper to improve the NUnit runner’s support for nested classes, and I’ll follow their lead.

Removed support for 5.1:

I started this project with ReSharper 4.1. I make that 7 releases (4.1, 4.5, 5.0, 5.1, 6.0 and the current 6.1.1 and 7.0). Going forward, I’m only going to support the last two major releases – i.e. 6.x and 7.x, so that means the current release has binaries for 6.1.1 and 7.0 RTM. You can still download the binaries to any of the old releases, and the source is still available in the repo, but they’re not going to see any new development.

So, there you have it. Go download it. If there’s a problem, report it at CodePlex, or give me a shout on Twitter.

Tags: , ,


xunitcontrib-resharper 0.6 - Theory support!

by Matt 5. July 2012 10:09


Yes, the latest release of the xUnit.net ReSharper test runner provides much better Theory support (theories are xunit’s parameterised tests). Up until now, the support has been so poor (i.e. non-existent) that it’s not always been possible to tell if a theory has failed, and if it has, which one did and why. It’s been the biggest issue and most requested feature since the project began.

Well, that’s all in the past now. Here’s the latest version running some Theory tests:


Yep, theories are now displayed as child nodes under the parent method, as you’d expect.

It provides support for any kind of Theory – data from attributes, properties, spreadsheets, databases and any other kind of custom Theory or multiple row producing custom Fact-based attribute you might like to write. It also works with dotCover to provide code coverage. (I’ll update the standalone dotCover runners shortly)

This release works with 6.1 and the current build of 7.0 (that is build 82). The download does not include runners for 6.0, 5.1 or earlier versions of ReSharper. Those versions are packaged up in an archive release. The idea is to only support the latest versions of the last two major releases of ReSharper.

As ever, download the release (PLEASE don’t forget to unblock the dlls), and if there are issues, let me know on codeplex or via twitter – @citizenmatt.

Tags: , ,

JetBrains, plugins + me

by Matt 20. June 2012 06:35

I've just checked, and I've been working on the xUnit.net ReSharper test runner for nearly 3 and a half years. I've always enjoyed writing tools and plugins, possibly more than server side or front end coding, the stuff that's paid the bills. I like building tools that help make people more efficient, and then gets out of the way. I love the idea of tweaking or extending a tool and making it to do new and interesting things.

One of my all time favourite tools is ReSharper. I’m a huge fan, use it constantly, and don’t know how I managed to develop without it. I know ALL the keyboard shortcuts and everything.

Which means that I'm VERY excited that from the start of July, I'm going to be part of JetBrains' .net developer evangelism team, working with Hadi, Jura and the others. Specifically, I'll be focussing on the extensibility features of ReSharper and the other .net tools (dotCover, dotTrace, dotPeek - yep, these guys are extensible too).

In other words, I'll be working on plugins.

Or rather, working with the whole plugin ecosystem. While it will involve writing, maintaining and publishing plugins, it's also a very community facing role. I'll be involved in, and provide support for, the existing community of plugin authors - helping out with issues and updates, gathering feedback, improving the ReSharper SDK. I’ll be encouraging new plugins by working on blog posts and tutorials, and participating in JetBrains events and community gatherings. Everything I can to improve ReSharper as a platform.

In some respects, this is quite a departure to what I've done in the past. I've worked in C++ and C#, a bit of Java and even some Flash (don't tell anyone). I've been a developer, team lead or architect in small code shops and several big enterprises. Now I'm going to be working from home (which beats the London commute I've been doing for the past few years), with a very social, community focussed role, working full time on the kinds of things I've been tinkering with in my spare time.

I'm massively looking forward to it.

Tags: , , , , ,

Running xUnit.net tests in dotCover 2.0 EAP

by Matt 18. April 2012 10:03

JetBrains recently released an EAP of dotCover 2.0. Up until now, dotCover required ReSharper to enable code coverage of unit tests – it integrated itself nicely in the ReSharper menus and UI, and would provide a coverage analysis of the code executed during a test run.


Starting with this 2.0 EAP, you no longer need ReSharper installed. DotCover provides support for its own test runner and test runner plugins. In fact, it comes bundled with a port of the unit test support and framwork from ReSharper, so you won’t actually be able to tell the difference.

Which means, of course, a new build of xunitcontrib to provide support for dotCover. This is a separate release to the ReSharper support – while the products have (pretty close) source code compatibility, there isn’t (yet?) binary compatibility.

Installation instructions are as simple as for the ReSharper plugin:

  1. Make sure you UNBLOCK the downloaded zip file (right click –> properties –> Unblock).
  3. Extract the zip and copy the xunitcontrib.runner.dotcover folder into C:\Program Files\JetBrains\dotCover\v2.0\bin\plugins (you will probably need to create the plugins folder)
  4. Restart Visual Studio.

Once installed, xUnit.net tests should be recognised, and you should now be able to run, debug and cover xunit tests.

Note that you DO NOT NEED this plugin if you have ReSharper (and the appropriate xunitcontrib version) installed. By default, dotCover still integrates with ReSharper, if it’s available.

If, however, you want to see what all the fuss is about, you can disable ReSharper’s unit test implementation, and use dotCover’s. Simply go to the ReSharper –> Options –> Unit Testing, uncheck “Enable Unit Testing”, and then go to the dotCover –> Options –> ReSharper Integration and uncheck “Hide all actions and tool windows related to unit tests in dotCover and use ReSharper’s implementation”. You will now be using the dotCover unit test implementation.

It’s going to look very familiar.


(And don’t forget to check out the “Colour Scheme” options if you’ve customised your Visual Studio colours. “Dark” makes SUCH a difference)


Tags: , ,

xunitcontrib-resharper 0.5.2 - ReSharper 6.1 support

by Matt 22. December 2011 12:35

ReSharper 6.1 has just been released. It’s better than 6.0, and it’s a free download, so go and get it. And then download the latest version of xunitcontrib to get support for your xUnit.net tests.

This version contains plugins for ReSharper 6.1, 6.0 and 5.1.3. I plan to only support the most recent revisions of the last two major versions, so that means 6.1 and 5.1.3. So, I’ll be moving 6.0 into the archive release very soon.

This is a small update. The important news is the support of ReSharper 6.1, but there was still chance to squeeze in a new feature.

ReSharper 6.0 added a nice little feature for running the derived instances of test methods defined in an abstract test method. To explain that a bit better, assume you’ve got an abstract base class that contains a test method:

public abstract class AbstractBaseClass
    public void TestMethod()
        // ...

and then a derived class:

public class DerivedClass : AbstractBaseClass
    // ...

ReSharper 6.x provides support for adding a marker next to AbstractBaseClass.TestMethod that gives a drop down menu allowing you to run TestMethod in one or all of the derived test classes. Something like this:


Unfortunately, 6.0 doesn’t provide enough support to get the naming correct everywhere in the UI. You can either specify the name as DerivedClass.TestMethod (which makes most sense when used in this drop down) or BaseClass.TestMethod (which reads better in unit test explorer and runner windows).

I’ve chosen to use BaseClass.TestMethod, because I think it’s more useful that the names are correct in the unit test explorer/runner windows (the built in nunit plugin uses DerivedClass.TestMethod). Annoyingly, this means the drop down isn’t terribly helpful:


The good news is that this all works correctly in 6.1.

So, what are you waiting for? Go upgrade.

Tags: , ,


xunitcontrib-resharper 0.5–fixes, fixes, fixes

by Matt 28. November 2011 12:49

I’ve just pushed a new version of xunitcontrib up to codeplex. Well, two new versions.

There’s the stable, ReSharper 6.0 and 5.1.3 version, and then the latest build for the 6.1 EAP. Both have had the same bunch of fixes made:

On top of that, the EAP version should now work properly in Visual Studio 2011.

The two interesting ones are the fixed dotCover support and the getting partial classes working. While the last release would show methods as highlighted, it was missing support for Show Covering Tests. This allows you to right click on any method and when select the Show Covering Tests will list all tests that call into that method. More niceness from JetBrains.


Getting partial classes working is important if you want to use Microsoft’s Pexframework. This tool will generate unit tests for you based on the code your write. These generated tests live in partial files, which now work in 6.0 and 6.1.

And finally, a quick word about versions. I intend to support that latest released revision of the last two major versions of ReSharper. In other words, I’ll support 6.x and 5.x by releasing versions for 6.0 and 5.1.3. Once the 6.1 EAP is fully baked, I’ll only support 6.1 and 5.1.3. I’ve added another release to hold these older versions.


Month List


Comment RSS