xunitcontrib 0.3 – external annotations and live templates

by Matt 9. July 2009 20:33

As part of the latest release of xunitcontrib, I’ve added two new features, support for ReSharper’s External Annotations and Live Templates.

External Annotations

ReSharper performs some static analysis on your code, and displays a blue squiggly error indicator over anything it thinks will be a problem. One great example is possible System.NullReferenceExceptions – using an object which may be null.

Possible System.NullReferenceException is flagged 

One thing you might notice from that screenshot is that we’ve got an Assert.NotNull in there. This method throws an exception if the “otherFlagsAttribute” is null, meaning we can’t get a NullReferenceException. But ReSharper doesn’t know that, and so we still get the warning.

As I understand it, ReSharper can only analyse your source code, and not any compiled assemblies. So how do we tell it that Assert.NotNull ensures that we don’t get a NullReferenceException?

Well, ReSharper makes available an assembly called JetBrains.Annotations.dll that provides a number of attributes that can handle this scenario (plus a whole bunch more, like not null parameters, or marking parameters as string format specifiers). Handy, but a bit intrusive to your code, and not much use if you’ve not got the code available to change.

And that’s where External Annotations come in. You can now unobtrusively apply these attributes to the code through xml files. (And the ReSharper guys have very nicely produced files for the entire .net framework…)

So, that’s what this feature is. It adds the xml file telling ReSharper that xunit’s Assert.NotNull is an assert method that will stop execution of the method if the parameter is null. Similarly, there’s annotations for Assert.Null, Assert.True and Assert.False.

(Interestingly, it’s easy to concoct an example for NotNull and also for True or False (simply create an expression that is always false or always true) but I can’t create an example for Assert.Null. Answers on a postcard, please…)

To install, simply drop the xunit.xml file into ProgramFiles\JetBrains\ReSharper\v4.x\bin\External Annotations.

Live Templates

Live Templates are very much like Visual Studio’s built in snippets feature (I guess I ought to look into porting this to snippets…) but with a bit more oomph. They allow you to type an identifier that automatically expands into something useful. For example, the Live Template provided expands the word “fact” into:

"fact" live template expanded as a fact method

Which has the method name highlighted ready to edit, and when you hit enter, places the cursor into the body of the method. It also automatically adds any using statements necessary.

These are very useful, especially with the smart Intellisense shown for the assert templates:

Assert.Equal Live Template with smart intellisense

I’ve actually released two versions of this. One which uses acronyms such as “ae” and “ann” to represent “Assert.Equal” and “Assert.NotNull”, and another that uses similar acronyms, but beginning with “x” (“xe”, “xnn”). This is to avoid a collision with Live Templates for a certain other unit testing framework that I use.

Here’s a list of the Live Templates included (the words wrapped in dollar signs like $this$ are macros, and are quickly edited by pressing tab, and usually display some sort of useful Intellisense):

Shortcut #1 Shortcut #2 Expands to
ae xe Assert.Equal($expected$, $actual$)
at xt Assert.True($value$)
af xf Assert.False($value$)
ac xc Assert.Contains($expected$, $actual$)
an xn Assert.Null($value$)
ann xnn Assert.NotNull($value$)
athr xthr Assert.Throws<$T$>()
aiaf xiaf Assert.IsAssignableFrom<$T$>($object$)
ait xit Assert.IsType<$T$>($object$)
fact fact Creates an attributed test/fact method
fa fa [Fact]
theory theory Creates an attributes theory method
ta ta [Theory]
ida ida [InlineData()]

Note that not all of the assert methods are there. Only the most common have been added. And I haven’t added Assert.Same, namely because “as” is a poor acronym to go for.

This is a little trickier to install. In Visual Studio, go to ReSharper –> Live Templates, then click the button to “Mount storage” and navigate to and select the live template file of your choice (“ae” or “xe” format).

If you mount the file, then any changes are saved back to this file. If you import it, then any changes are kept in a copy of the config file. I prefer mounting the file, as it gives me more flexibility to change the templates and pass the changes around.

So that covers external annotations and live templates. As ever, if you have any bugs or comments, please use the Issues and Discussion pages of the CodePlex project site. And feel free to get in touch on Twitter, too (via @citizenmatt or #xunitcontrib).

Tags: ,

My favourite xunitcontrib release yet

by Matt 9. July 2009 20:18

Yep. It’s xunitcontrib release time again, and this one’s a pretty major one. If you use the ReSharper runner, you really want to download this one.

(If you want to cut to the chase and just get the binaries and install instructions, then check out the release page on CodePlex.)

As usual, it’s a ReSharper focussed release, with improvements to the test runner and the addition of External Annotations and Live Templates. I’ll look at these last two in another post, because I want to focus on the big ticket item today – the runner.

So what’s the big news?

Well, if you’ll forgive a little enthusiastic hyperbole:

A self congratulatory twitter post announcing version independent test runner

Version independence.


Up until now, the ReSharper test runner would run your tests with the 1.1 released version of xunit.dll only. If your test project is using a different version, such as the new CTP of 1.5, you’ll get a cryptic error message, and no tests would run.

One significant rewrite later, and it’s now using the same version independent runner API that the console and msbuild runners use.

In short, you can now use any version of xunit you like.

Cool, and very much a big deal.

But rather embarrassingly, the bit which pleases me most is a simple side effect.

One of the nice things that xunit does is randomise the order in which tests are run. And because I’m now letting xunit handle everything, we get that for free:

The ReSharper test runner running tests in a random order

(I really should have done that as an animated gif; it’s nice to see in action.)

So that’s the Big New Feature. What else is there? This is a small one, but I like it. Nested exceptions are now formatted nicer. Before:

nested execeptions formatted as a single exception


nested execptions formatted as separate exceptions

And give or take a couple of things, that’s pretty much it for user-visible changes. But there are a couple of other things I want to point out though.

Firstly, that breathless Twitter post above came from a deep dive into the ReSharper test runner framework. Amusingly (after having worked on this project for the past 4 or 5 months) I finally understand how all of this works. And I’ve been heavily commenting the code to explain what’s going on. I hope it can be of use to someone else who wants to write a test runner.

(Yes, that’s right, I just made code comments a feature.)

I also have another post brewing about my developer conscience kicking in and pointing out that I could actually write unit tests for the newly restructured code. (Short version: TDD rocks + I finally get BDD.)

But it’s not all roses. There’s still no installer, and there’s truly terrible support for theory style tests (and I mean terrible. You’ll see output and exceptions, but not together. And you won’t know which test row caused the output, especially since they’re now run in random order too)

So what’s next?

Well, in part, that’s up to you. Check out the Issues on CodePlex and vote for what you want. Leave a comment here or on Twitter (via @citizenmatt or #xunitcontrib). But I really want to do something about those theory tests. I’m not sure what yet, though.

Right, that’s enough. Go and download it already.

Tags: ,

Announcing xunitcontrib 0.2 – Resharper 4.5 support

by Matt 14. April 2009 17:24

I’ve just released 0.2 of the xunitcontrib project (actually thought I’d released it over the weekend. Hadn’t pressed the right button. Oops – it’s there now.)

This release is primarily to support the RTM release of Resharper 4.5 (with a couple of smaller bug fixes too). It runs just like the old one, but with one added feature specifically for 4.5 – support for Resharper’s Solution Wide Code Inspections. When this is turned on, Resharper will highlight any public methods that are not used as part of a project. And of course, test methods are public methods that are not used by any other part of a project. One implementation of the wonderfully named IsTestStuff, and we’re all sorted.

Check out the downloads page for more details, and instructions on how to install.

And if you’ve got any comments, issues or ideas, please post them to the CodePlex site or leave a comment here, or via twitter, either directly to me (@citizenmatt) or via the #xunitcontrib hashtag.

Tags: ,


Month List


Comment RSS