ReSharper runner for xUnit.net 2.0

by Matt 2. May 2014 05:55

Here’s two nice things to end a Friday with.

Firstly, I’ve just pushed a new, 2.0.0 ALPHA pre-release version of the xUnit.net runner for ReSharper to the extension gallery. It adds support for xUnit.net 2.0 beta 2. You can get it from ReSharper’s Extension Manager. Make sure you select the “pre-release” drop down in the Extension Manager first.

Secondly, the project is no longer hosted on CodePlex, but has moved to GitHub, and lives under the xunit org. All issues and pull requests should go there please.

xUnit.net 2

You should consider this an alpha version, because, well, it is. It works, I’ve got a bunch of tests to prove it, but hey, it’s all new and different, so if you find any issues, please, please, please, let me know (twitter or, even better, GitHub. Did I mention the project’s moved?) In fact, any kind of feedback is appreciated, good or bad.

Thanks to the excellent support built into xunit2 by Brad and Jim, the new runner can handle both your xunit1 and xunit2 tests. Everything works as you’d expect – tests are highlighted in the editor, and you can run and debug them as normal.

However, this is an alpha build. There are couple of important caveats:

  • Parallelisation is DISABLED in this release. This is due to how xunit2 handles errors. Currently, there is no way for me to gracefully handle errors thrown when a class or collection fixture throws an exception. Hopefully this can be resolved, and I’ll enable parallelisation.
  • Test discovery in the editor is currently handled by xunit1. This needs migration, but is a complex beast. It means that currently, anything more interesting than plain Fact or Theory based tests might not work. Test discovery in the runner is handled by xunit2, hence the weasel word “might”. Fixing this is my next priority.
  • It currently only supports 8.2, as this is what I’ve been using to build and test. I’m not sure yet if I’ll add support for 8.1 back. Leave feedback on GitHub if you really want it – but leave a reason, too. After all, 8.2 is a free upgrade!
  • Annotations and live templates haven’t been updated yet, or even tested. They might work, they might not – so your methods might not be highlighted as in use, and the templates might not work (theory definitely won’t – it’s in a different namespace)

Moving to GitHub

I think it’s fair to say that xunitcontrib had a good run on CodePlex, but, well, GitHub is better. It’s a much nicer experience, everything is just easier, and well, GitHub won.

A very nice feature on GitHub is organisations. Brad and Jim have very generously invited the ReSharper runner to live under the xunit org. It doesn’t change the ownership, copyright or license of the project (xUnit.net is an Outercurve project, but the ReSharper runner is mine), it’s simply a nice place to live.

The name of the project has also changed. It always was more of a ReSharper project than a contrib project, so the name is now “resharper-xunit” to reflect this. I’ll still be using “xunitcontrib” as the name of the ReSharper packages, though.

What’s next?

Time to improve on the support. First item on the list is proper xunit2 based test discovery. I’ve opened a PR to track xunit2 progress, and more issues to track new functionality.

Please test it! Get it from the extension manager, and put it through its paces. And if you find any problems, report them!

Tags: , , ,

xunitcontrib

xunitcontrib-resharper 0.7 - ReSharper 7.1 and stuff

by Matt 20. November 2012 03:39

ReSharper has just released 7.1, so I think it’s only fair to update the xUnit.net test runner to support it.

And it would be mean not to throw in a couple of bug fixes while I’m at it.

Firstly, there’s a fix to support filtering out test usages when using ReSharper’s Find Usages command. Frankly, I thought this was working ages ago, but it looks like I broke it some time back. Sorry folks.

When in the Find Results window, displaying the usages of a type, method, property, etc. you can filter out different types of usage – read/write, invocation, usage in attributes or documentation, and, what’s interesting here, usages in tests. This means you can hide any usage of a type from a test method or class. Simply toggle the “Show Unit Test Usages” menu item in the filter, and the Results window will hide and show the test usages (note that the test usages are using the test icon that appears next to the test method in the code editor).

FilterTestUsages

The second bug fix relates to Theory data rows. xUnit.net uses class and method names as a means of identifying a test. This is guaranteed to be a unique value, as the test method doesn’t run more than once. For Theory parameterised tests, xunit uses the class and method name, but also uses a display name, which includes the parameter values, such as “ShouldBeUppercase(s: SAUSAGES)”.

There is a problem here. If you have two rows of data with the same value (“SAUSAGES”), the test runner treats these as the same test. This is a case of “doctor, doctor, it hurts when I do this”. Don’t do this. You’re passing in the same data, you’ll get the same results, it doesn’t matter if it looks like there’s only one test.

Or so I thought.

What happens if the data being passed in is an array of strings? xunit will call ToString on the parameter value, and come up with a display name like “ShouldBeUppercase(s: System.String[])”. Clearly, there are going to be collisions here. So now, the test runner keeps a track of names of tests it’s already run. If it tries to run the same test again, it changes the name, appending a call number, e.g. “ShouldBeUppercase(s: System.String[])” and “ShouldBeUppercase(s: System.String[]) [2]”, “… [3]” and so on.

RepeatedTheoryNames

And that’s it for this release. I smell a 1.0 version coming, finally. There are a couple of big ticket changes I want to make first. I want better support for PropertyDataAttribute (IntelliSense, find usages, ctrl-click navigation, etc) and InlineDataAttribute, and there’s a good memory optimisation I need to make for editing a large test file. If there’s anything else missing or broken, now’s a good time to let me know!

As ever, download it from Codeplex, install it with the handy batch file, let me know any issues.

Tags: , ,

xunitcontrib

xunitcontrib-resharper 0.6.1 - mostly bug fixes

by Matt 2. August 2012 02:42

Seems like this blog is becoming just an announcement service for new versions of my xUnit.net test runner for ReSharper. Bit of a maintenance release, this one. Here’s what it’s in:

RTM support for ReSharper 7.0 RTM:

The 0.6 release worked just fine with the RTM, but here’s the official build.

Bug fixes:

  1. The wrong dll was used when you switched between Debug and Release. Yikes!
  2. Adding a method to the end of a class, and then trying to run all tests in a class could cause the whole class to fail, as it tried to run a method called “???”
  3. If you used non-public test methods, you couldn’t double-click to navigate in the test sessions or test explorer views
  4. Exceptions were thrown when trying to parse a source file that contained static public properties and had a build action set to None. Rather a specific one, that.

The most interesting change is for placing tests in nested classes. When displaying a test method in the UI, it’s usually displayed as the class’s short name, e.g. For “Name.Space.ClassName”, you see “ClassName”. This scheme fails when you have nested classes, and multiple nested classes share a name, but have different parent classes. Like this:

public class AddressTests
{
  public class ToStringTests
  {
    [Fact] public void ShouldReturnFormattedAddress() { /* ... */ }
  }
}

public class PersonTests
{
  public class ToStringTests
  {
    [Fact] public void ShouldReturnForenameAndSurname() { /* ... */ }
  }
}

(Check out Phil Haack’s post that nicely describes why you might want to do this). The problem is that the test runner would display this:

ambiguous_nested_classes

making it very hard to distinguish which test was which. Now they display like this:

unambiguous_nested_classes

In other words, nested classes include the name of their parent class in the UI. I’m not exactly enamoured of the naming format; using a “+” to separate the class names isn’t very well known outside of Relection, but at least it gives you the chance to know which test is which. It might change in a future version - there’s an outstanding bug for ReSharper to improve the NUnit runner’s support for nested classes, and I’ll follow their lead.

Removed support for 5.1:

I started this project with ReSharper 4.1. I make that 7 releases (4.1, 4.5, 5.0, 5.1, 6.0 and the current 6.1.1 and 7.0). Going forward, I’m only going to support the last two major releases – i.e. 6.x and 7.x, so that means the current release has binaries for 6.1.1 and 7.0 RTM. You can still download the binaries to any of the old releases, and the source is still available in the repo, but they’re not going to see any new development.

So, there you have it. Go download it. If there’s a problem, report it at CodePlex, or give me a shout on Twitter.

Tags: , ,

xunitcontrib

xunitcontrib-resharper 0.5.2 - ReSharper 6.1 support

by Matt 22. December 2011 12:35

ReSharper 6.1 has just been released. It’s better than 6.0, and it’s a free download, so go and get it. And then download the latest version of xunitcontrib to get support for your xUnit.net tests.

This version contains plugins for ReSharper 6.1, 6.0 and 5.1.3. I plan to only support the most recent revisions of the last two major versions, so that means 6.1 and 5.1.3. So, I’ll be moving 6.0 into the archive release very soon.

This is a small update. The important news is the support of ReSharper 6.1, but there was still chance to squeeze in a new feature.

ReSharper 6.0 added a nice little feature for running the derived instances of test methods defined in an abstract test method. To explain that a bit better, assume you’ve got an abstract base class that contains a test method:

public abstract class AbstractBaseClass
{
    [Fact]
    public void TestMethod()
    {
        // ...
    }
}

and then a derived class:

public class DerivedClass : AbstractBaseClass
{
    // ...
}

ReSharper 6.x provides support for adding a marker next to AbstractBaseClass.TestMethod that gives a drop down menu allowing you to run TestMethod in one or all of the derived test classes. Something like this:

image

Unfortunately, 6.0 doesn’t provide enough support to get the naming correct everywhere in the UI. You can either specify the name as DerivedClass.TestMethod (which makes most sense when used in this drop down) or BaseClass.TestMethod (which reads better in unit test explorer and runner windows).

I’ve chosen to use BaseClass.TestMethod, because I think it’s more useful that the names are correct in the unit test explorer/runner windows (the built in nunit plugin uses DerivedClass.TestMethod). Annoyingly, this means the drop down isn’t terribly helpful:

image

The good news is that this all works correctly in 6.1.

So, what are you waiting for? Go upgrade.

Tags: , ,

xunitcontrib

xunitcontrib support for ReSharper 6.0 beta

by Matt 14. June 2011 09:18

So, ReSharper 6.0 has gone beta. And that means we need a new build of the xUnit.net test runner plugin. The latest release on codeplex works nicely with the beta version, and I’ll be aiming to keep it up to date as the nightly builds continue.

There are a couple of minor changes with this release, changes that are in the source for the 5.1 version, but haven’t yet been released (that’s the missing 0.4.2 version for you eagle-eyed folks). You can read more in the release notes.

PLEASE READ THE INSTALLATION INSTRUCTIONS!

There is currently a bug/regression in ReSharper that means you can’t just throw all of the files in the plugins folder. Instead, you must copy the xunit.runner.utility.dll and the xunitcontrib.runner.resharper.runner.6.0.dll files into the C:\Program Files\JetBrains\ReSharper\v6.0\bin folder. If you copy all of the files from the zip (including the plugins folder) straight into the bin folder, you’ll be fine. I’m hoping this will be fixed by RTM.

Go download!

Tags:

xunitcontrib

Running xunit tests in Silverlight–xunitcontrib-silverlight

by Matt 14. December 2010 12:14

If you want to write unit tests in Silverlight,  it’s a pretty solid bet that you’ll be using using Jeff Wilcox’s unit testing framework that ships with the Silverlight Toolkit. It’s essentially a port of mstest, combined with a Silverlight based test runner. Handily, the test runner allows for unit test providers, allowing us to plug in a different test framework.

And that’s exactly what the latest release of xunitcontrib is – a port of xUnit.net, and a provider for the unit testing framework.

image

It’s a pretty faithful port of xunit (i.e. with as few changes to the source as possible), and includes the majority of the features that you’ll require day-to-day for testing, but it’s also running on a different platform and in a different execution environment to the desktop CLR version of xunit, so THERE ARE DIFFERENCES.

I strongly suggest checking out what’s different or not included in the xunit port, as well as the details of the unit testing framework integration.

In the mean time, you can download the release from codeplex. It supports Silverlight 4 using the April 2010 toolkit and Silverlight 3 using the semi-official build based on the April 2010 toolkit (Windows Phone 7 support is coming in the next release). Here’s the quick run down on how to use it:

  1. Create a Silverlight application
  2. Add references to Microsoft.Silverlight.Testing.dll and the Silverlight-appropriate versions of xunit-silverlight.dll, xunit.extensions-silverlight.dll and xunitcontrib.runner.silverlight.toolkit.dll
  3. In App.xaml.cs, replace Application_Startup with:
          UnitTestSystem.RegisterUnitTestProvider(new XunitContrib.Runner.Silverlight.Toolkit.UnitTestProvider());
          RootVisual = UnitTestSystem.CreateTestPage();
  4. Then just add tests, and run the application

Alternatively, if you’ve installed the latest Silverlight Toolkit, you’ll get a unit testing project template in the File->New dialog that makes this a little easier, although you’ll still need to add the xunit/xunitcontrib references and register the xunitcontrib provider (if you don’t register it, you’ll get no tests!)

As ever, let me know, either via codeplex or on twitter (@citizenmatt) if there are problems or questions.

Tags:

xunitcontrib | Silverlight

Announcing xUnit.net Contrib

by Matt 21. February 2009 19:21

I’ve just flicked the switch and published the xunitcontrib project on Codeplex. My first Open Source project. How nice.

Hopefully, xUnit.net will need no introduction. It’s a very nice unit testing framework; minimalist, idiomatic and opinionated.

The only problem was, the ReSharper runner kept crashing.

So after getting the source and getting a little carried away fixing a few other bugs, I wanted to contribute the changes back. Only to find that the project leads, Brad Wilson and James Newkirk, were looking for someone else to maintain the ReSharper runner (it’s a moving target for them, and I believe neither of them use it).

And so here we are with the xunitcontrib project.

Currently, it’s just the ReSharper runner, but I want that to change quite quickly. I’ve created a Roadmap page on the project site to list some ideas. I want to keep updating the ReSharper runner, and I want to add a nant runner, but I really want to get to the meat of the project – the xunitcontrib.dll.

If you look at xunit, it’s split into 3 parts, the core, xunit.extensions and samples. I see this project as sitting between xunit.extensions and samples. Second party extensions, if you will. Extensions that can afford to have a narrower focus than the framework, or perhaps they don’t quite adhere to some of the core sensibilities of xunit (the ordered tests in the samples is a good example of this – xunit randomises tests by default).

Another thing I’d like to see is a way to ease migration towards xunit. I want to use xunit at work, but the projects I’m working on have too much of an investment in nunit, and the inertia this has built up is rather daunting. I’d like to make it easy to just start using xunit. That said, I’m not too sure what this would look like.

So if you’ve got any comments or suggestions, then please post them to the Discussions page or raise a work item in the Issue Tracker. Or just go grab the download for some ReSharper test runner goodness.

xunitcontrib.runner.resharper

Tags:

xunitcontrib

Month List

RecentComments

Comment RSS