The first post in this series described the three major testing frameworks in .NET: MSTest, NUnit, and xUnit. Here I’ll be describing how to get started writing actual tests in each of them.
Marking a class as containing a test
Some test frameworks require you to mark a class as a test container.
MSTest
A test in MSTest must be contained in a test class. A test class must be a public class with a default constructor and must be marked with the [TestClass] attribute. It must not be abstract or static nor have generic parameters. The test explorer and test runners will fail to pick up a test class that does not exactly meet any of these criteria; any tests contained therein will simply not appear in the list of tests to be run. That is, except for the “default constructor” requirement; in that case, it will detect the test class and methods contained therein but fail the tests at runtime with the following exception:
Starting a new test project in Visual Studio will give you a file called UnitTest1.cs which contains an example of such a class
NUnit
A test in NUnit must be contained in a test class which is called a test fixture. The requirements for a test fixture may be found here, but I will summarize them anyway. It must be marked with the [TestFixture] attribute. Like MSTest, it must not be abstract nor have generic parameters. Unlike MSTest, the test fixture need not be public. (it can even be a private nested class of another class) and it may even be static. If you try marking an abstract class as a test fixture, it will simply not discover it. If you try marking a generic class as a test fixture, the test run will simply not run (the tests won’t fail; it simply won’t run) but you will get the following warning.
[6/23/2018 1:52:20 PM Warning] SetUp failed for test fixture NUnitTest1.Class1
[6/23/2018 1:52:20 PM Warning] Fixture type contains generic parameters. You must either provide Type arguments or specify constructor arguments that allow NUnit to deduce the Type arguments.
The warning hints at the fact that there may indeed be a way to use a generic test fixture. This is true, and we will see how in Part Three of this series.
I also said nothing about requiring a default constructor, because that is also not required. Again, we will see how to utilize this feature in Part Three.
xUnit
xUnit requires the test class to be public, non-abstract, and non-generic. Until we get to Part Five: Advanced Features, it is simpler to say that a default constructor is also required. However, unlike the other two frameworks, an xUnit test class does not require any attributes to be applied on the class. If a test is discovered inside a non-public class, xUnit will give a compile-time error telling you what to fix.
Marking a method as being a test
It is useful for a test to be able to call helper methods often defined in the same class as the test itself. As such, it is impossible for the framework to tell which methods are tests and which are simply helpers unless you give it a little guidance.
MSTest
A test in MSTest is a method marked with the [TestMethod] attribute. It must be a public instance method which takes no parameters. It must also return void, unless the test is async in which case it must return System.Threading.Tasks.Task. If a method decorated with [TestMethod] does not match this signature, it will be omitted from the test run and the following warning output by the test discoverer.
UTA007: Method TestMethod1 defined in class MSTest1.UnitTest1 does not have correct signature. Test method marked with the [TestMethod] attribute must be non-static, public, return-type as void and should not take any parameter. Example: public void Test.Class1.Test(). Additionally, if you are using async-await in test method then return-type must be Task. Example: public async Task Test.Class1.Test2()
You can get the output from the test discoverer by going to the Output window in Visual Studio and changing the “Show output from:” option to “Tests”.
NUnit
You can make a test class’s method a test by marking it with the [Test] attribute. It must be a public method, but it may be instance or static. Just as with MSTest, the method may return async Task instead. You can also give it an expected return value, like the following.
It is also possible to have parameters in the method. See Part Three of this series for more information on that.
xUnit
A test can be created in xUnit by marking the method with the [Fact] attribute. A fact must be a parameterless method. It may return a value if you wish, but the value will not be consumed. It may be static or instance.
Setting Up and Tearing Down
Modern unit tests are frequently written in what is known as Gherkin format, or “Given/When/Then”. For example…
Given a logged-in customer
When the customer adds an item to their cart
Then the item is now in the cart inventory
When testing this requirement, there are likely to be other similar situations. Consider this one…
Given a logged-in customer
When the customer adds an item to their cart
Then the checkout price includes the new item
Or this one…
Given a logged-in customer
When the customer logs out and back in
Then the customer’s cart has not changed
It is clear that the code to write these three tests will have some overlap. It may be quite a bit of overlap. In particular, they all have the same Given and will therefore probably have the same setup phase to the test. You can think of the setup as the “Arrange” in the Arrange/Act/Assert pattern. It is likely that all the tests related to this particular scenario will be grouped into one class. Therefore, test frameworks provide a way to automatically run some setup code before tests. In case the tests claim expensive resources like file locks or database connections, they also provide a way to automatically run cleanup code after tests. We will now look at the mechanisms for achieving this in our three frameworks.
MSTest
Each of the setup and teardown code for MSTest is indicated by placing an attribute on a method. The method will then have a companion test. Any console output recorded during the method’s execution will be logged as if it came from the companion test. See the table below.
| Attribute | Companion Test | Runs | Signature |
|---|---|---|---|
[TestInitialize] |
Each test in its test class | Before companion | public void Init() |
[TestCleanup] |
Each test in its test class | After companion | public void Cleanup() |
[ClassInitialize] |
First test in its test class | Before companion | public static void Init(TestContext context) |
[ClassCleanup] |
Last test in its test class | After companion | public static void Cleanup() |
[AssemblyInitialize] |
First test in its assembly | Before companion | public static void Init(TestContext context) |
[AssemblyCleanup] |
Last test in its assembly | After companion | public static void Cleanup() |
Three additional notes are worth including.
First, the signature must be an exact match except for the method name which may be any valid C# identifier.
Second, MSTest runs each test against its own instance of the test class. As such, setup that must be done before each test could be done in the class’s default constructor as easily as the TestInitialize method. Console output in that method is recorded with its companion test. However, convention and clarity are better served by using the idiomatic TestInitialize attribute.
Third, the TestContext argument records various details about the companion test. One could keep a reference to it if desired, but note that it will not be updated as other tests are executed. For more information on TestContext see Part Five in this series.
NUnit
NUnit’s initialize and cleanup perform similarly to MSTest’s. The following is a translation guide. Unlike MSTest, any of these methods may be declared as static or instance.
[SetUp]: analagous to[TestInitialize].[TearDown]: analgous to[TestCleanup].[OneTimeSetUp]: analagous to[ClassInitialize]. Signature ispublic void Init().[OneTimeTearDown]: analagous to[ClassCleanup].
You may decorate a public non-static non-abstract class having a default constructor with the [SetUpFixture] attribute. Then the [OneTimeSetUp] and [OneTimeTearDown] methods inside that class will be run, respectively, once before the first test in the setup fixture’s namespace and once after the last test in the setup fixture’s namespace. Placing the fixture in the global namespace allows setup and teardown for the entire assembly.
Note that NUnit does not create a separate instance of the test fixture for each test inside it, so adding code to the constructor is not recommended here. Also, NUnit does not record console output made during the constructor.
xUnit
The designers of xUnit consider test setup and teardown to be something of an anti-pattern, leading to hard-to-maintain unit tests. As such, they recommend against using it. However, they do recognize that testing frameworks such as xUnit are used for integration testing as well, so they have included a few hooks.
As with the other frameworks, each test gets its own instance of the test class. Therefore, any setup that needs to be run once before each test in a test class should be run in that class’s default constructor.
If your test class needs to run a piece of code to release resources after each test in the test class, you may accomplish this by having your test class implement System.IDisposable and putting the cleanup code in the resulting Dispose() method.
xUnit does not support adding setup or teardown at the class, namespace, or assembly levels. Class-level setup code could be emulated by using the class’s static constructor; however, be aware that this is run at test build time, not at test runtime, so you won’t be able to step into it with the debugger, and it may lengthen your build process. Similar effects may also be achieved by using xUnit’s ClassFixture and CollectionFixture features, but those will be covered in Part Five of this series.
By default, xUnit runs tests in parallel. As such, console output is not recorded during any xUnit tests. The framework would have no way of distinguishing which test the output came from.
Organizing Your Tests
When following a strategy like TDD, one will end up with a large number of tests. At some point, it becomes necessary to organize them. The common test explorers have the ability to group tests by traits and to ignore tests marked in a given way. That is what we will explore in this section.
MSTest
[TestCategory]: This attribute’s constructor takes a single string parameter. You can use it to delineate categories of tests. For instance,[TestMethod, TestCategory("Issue 315")]marks the test as belonging to a category called “Issue 315”.[TestProperty]: This attribute’s constructor takes two string parameters that function as a key-value pair. You can use it to delineate categories of tests. For instance,[TestMethod, TestProperty("Issue", "315")]marks the test as having a property called “Issue” with a value of “315”.[Ignore]: A test marked with this attribute will be skipped by the test run. It takes an optional string parameter which should be a user-readable description of the reason for skipping the test.
NUnit
The NUnit attributes in this regard are entirely analagous to the MSTest ones. TestCategory becomes Category, TestProperty becomes Property (which can now take a double, an int, or a string as the value instead of just a string), and Ignore remains Ignore, although the parameter is not optional.
Unlike MSTest, however, these attributes in NUnit are not sealed. Therefore, they can be subclassed and extended. The category attribute in particular will automatically take the name of the attribute type by default, allowing easy extension.
xUnit
The Trait attribute corresponds to the Property and TestProperty attributes in the other frameworks. It is a key-value pair where both are strings. There is no analogue to Category in xUnit. A test may be skipped by modifying the attribute which declares the test like so: [Fact(Skip="Reason")].
Summary
We have now created our first tests in all three frameworks and can organize them effectively with a good understanding of where they will go and how they will be run. In Part Three, we will discuss data-driven tests in all three frameworks and start to get into the power of xUnit’s extensibility.
The Developing Coder 