Part I
Developers who have incorporated unit testing into their development process already know its advantages: cleaner code, courage to refactor, and improved productivity. Writing unit tests is easy; it is writing good tests that makes automated unit testing and test-driven development powerful tools. This article provides some basic unit-test patterns that help write good tests, and anti-patterns you should avoid. For beginners, Gil Zilberfeld has also provided a good introduction to unit testing
A great unit test will do the following:
- Fail only when a bug has been introduced
- Fail Fast, for instant feedback
- Easy to understand what scenario is being tested and needs fixing
To do so we follow some of the following unit-test patterns.
Modern tests contain three parts:
|
1 2 3 4 5 6 7 8 9 10 11 12 |
[TestMethod] public void GetCount_ItemCountIsZero_NoNewMessages() { //Arrange Mailbox mailbox = new Mailbox(); //Act var result = mailbox.GetCount(0); //Assert Assert.AreEqual("No new messages.", result); } |
- Arrange: setup everything needed for the running the tested code. This includes any initialization of dependencies, mocks, and data needed for the test to run.
- Act: Invoke the code under test.
- Assert: Specify the pass criteria for the test, which fails it if not met.
This pattern is very readable and consistent. It’s very easy to identify the different parts of the test and to create tests for other scenarios from a root test. Get more information about this pattern here
In our unit tests, we’d like to test the result of the method execution. Usually, the code results in a state change, which is exposed by the object. We can look at state using:
- Values directly returned from the method
- Values exposed through the object fields
- Values exposed through other methods or properties of the object
- Values that come from outside the object, for example, static state or a shared data structure.
Tests that assert these values are called state-based tests. Depending on the test framework of choice, you’ll find Assert APIs that let you compare these values to an expected set.
In this example, we’re using MS-Test framework’s Assert.AreEqual to compare a returned value from method.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
[TestMethod] public void GetCount_ItemCountIsZero_NoNewMessages() { //Arrange Mailbox mailbox = new Mailbox(); //Act var result = mailbox.GetCount(0); //Assert Assert.AreEqual("No new messages.", result); } |
State-based tests are usually more robust than their interaction-based counterparts. They don’t increase coupling between the test and code in the Assert part since the assertion is made on data that is exposed through public interfaces. There’s no risk of tests breaking because of refactoring the internal implementation of methods.
This also adds to their readability: The tests describe behavior through public interfaces, without exposing internal mechanisms.
Sometimes, tested methods do not expose their result directly. For example, void methods do not return a value. If they modified some exposed data, we can then check that the data has changed. Yet, if the tested logic calls methods on other dependencies, we still can’t examine actual results.
When this happens, we cannot simply assert on data, but we need to check that the method got called, maybe checking the passed arguments as well.
Our tested method, ReportIfSpam, checks if the message argument is spam, and then calls the static Administrator.SendEmail method. Neither method returns a value.
|
1 2 3 4 5 |
public void ReportIfSpam(Message message) { if (message.IsSpam) Administrator.SendEmail(message); } |
We’d like to test what happens in case of a spam message. Our pass criterion is that we called the SendEmail method, with the supplied message as argument.
(In this test, part of the setup is to fake calls to the Administrator class; we don’t want to send a real email every time we run the test).
Most test frameworks don’t support interaction-based tests. We use mocking frameworks, in this case, Typemock Isolator, for this purpose.
|
1 2 3 4 5 6 7 8 9 10 11 |
[TestMethod] public void ReportIfSpam_MessageIsSpam_SendEmailToAdministrator() { Mailbox mailbox = new Mailbox(); Message spamMessage = new Message(); spamMessage.IsSpam = true; // Set message to be spam Isolate.Fake.StaticMethods<administrator>(); // Fake sending the emails mailbox.ReportIfSpam(spamMessage); //Assert the call was made with the correct argument Isolate.Verify.WasCalledWithExactArguments(() => Administrator.SendEmail(spamMessage)); } </administrator> |
Interaction-based tests require knowledge about the internal implementation of the method. Since they increase coupling, they are less robust than state-based tests. In addition, to understand the test failure, the reader needs to know more about the implementation, rather than interface, reducing readability.
As a guideline, given the option, we’d prefer state-based tests over interaction-based tests.
Throwing exceptions is part of behavior we’d like to test: making sure exceptions are thrown when they should, along with correct information.
Testing thrown exceptions depends on your test framework, each uses a different sub-pattern. NUnit and xUnit allow us to use APIs, that combine the Act and Assert parts. For example, in NUnit:
|
1 2 3 4 5 6 7 |
[Test] public void ReportSpam_MessageIsSpam_ExceptionIsThrown() { Mailbox mailbox = new Mailbox(); Message spamMessage = new Message(); Assert.Throws<spammessageexception>(()=>mailbox.ReportIfSpam()); } </spammessageexception> |
In MSTest (and old versions of NUnit), we test exceptions with an ExpectedException attribute. This version leaves the Arrange and Act parts intact, but breaks the AAA sequence.
|
1 2 3 4 5 6 7 8 9 |
[TestMethod] [ExpectedException(typeof(SpamMessageException))] public void ReportSpam_MessageIsSpam_ExceptionIsThrown() { Mailbox mailbox = new Mailbox(); Message spamMessage = new Message(); mailbox.ReportIfSpam(); } |
Comparing the two, the first is more readable and focused. In complex tests, it’s easier to see what we test and if the test fails, it’s easy to see what caused it. When an attribute-based test fails, it’s not apparent where the error occurred and may require debugging to resolve.
It is a good practice to unit test only public interfaces. We usually test for exposed behavior, rather than specific implementation. Increased coupling between the test and the code, may result in the test breaking when the internal implementation has changed, while the exposed behavior hasn’t changed.
That said, there are still cases where we want to test an internal behavior. A classic example is a Balanced Binary Tree. In order to test that the tree is balanced, we will need to access a private interface. They may benefit from testing the algorithms directly, rather than through the object interface. Legacy code frequently includes code that needs testing but is not exposed through a public interface.
There are a couple of solutions that incur changes to the tested code, or collaboration between code and tests.
- Making changes to the tested code
- Explicitly change private methods to public
- Exposing internal methods using the InternalVisibleToAttribute.
- Accessing the internal members from the test
- Using reflection
- Using other tools
The following is an example of how to invoke a private member function in a test using Isolator.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
[TestMethod] public void PrivateGetCount_ItemCountIsFive_FiveNewMessages() { //Arrange Mailbox mailbox = new Mailbox(); //Act var result = Isolate.Invoke.Method(mailbox, "GetCount", new object[] { 5 }); //Assert Assert.AreEqual("New messages: 5", result); } |
When unit testing, often the tested code interacts with external dependencies. These dependencies can be other objects in the system, files, databases, frameworks or 3rd party libraries. In order to speed up our tests, and control the behavior of the dependencies, we would prefer to use mock objects.
Mock (or fake) objects can be hand-written or supplied by a mocking framework.
In the following example, the method under test GetCountFromProvider has a dependency on an IMessageProvider interface:
|
1 2 3 4 5 6 7 8 9 10 11 |
public string GetCountFromProvider(IMessageProvider MessageProvider) { if (MessageProvider.Count == 0) { return "No new messages."; } else { return "New messages: " + MessageProvider.Count.ToString(); } } |
IMessageProvider has a read-only Count property. In order to test the method in different cases, we’ll need to change the returned value from that property.
We can define a fake message provider object that we can control. We can set the Count value, which is part of the original IMessageProvider interface, using SetCount method:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
public class FakeMessageProvider : IMessageProvider { int count; public int Count { get { return count; } } public void SetCount(int newValue) { count = newValue; } } |
Then our test will look like this:
|
1 2 3 4 5 6 7 8 9 |
[TestMethod] public void GetCountFromProvider_ItemCountIsZero_NoNewMessages() { Mailbox mailbox = new Mailbox(); FakeMessageProvider fakeProvider = new FakeMessageProvider(); fakeProvider.SetCount(0); var result = mailbox.GetCountFromProvider(fakeProvider); Assert.AreEqual("No new messages.", result); } |
The test creates an instance of the FakeMessageProvider, initializes it, then passes it as an argument. While using handwritten mocks works, they get really complex, as the tested code grows complex. The result is hard to maintain fake code, which may also have bugs in it.
The solution is to use a mocking framework, like Typemock Isolator. Isolator creates the fake objects, and we need to specify the behavior of those objects. With Isolator, the test looks like this:
|
1 2 3 4 5 6 7 8 9 |
[TestMethod] public void GetCountFromProvider_FakeItemCountIsZero_NoNewMessages() { Mailbox mailbox = new Mailbox(); IMessageProvider fakeProvider = Isolate.Fake.Instance(); var result = mailbox.GetCountFromProvider(fakeProvider); Assert.AreEqual("No new messages.", result); } |
Sometimes, however, our code depends not only on objects that are passed as arguments, or set through the tested code interface. For example, if our code depends on static data:
|
1 2 3 4 5 6 7 8 9 10 11 |
public string GetCountFromCenter() { if (MessageCenter.Count == 0) { return "No new messages."; } else { return "New messages: " + MessageCenter.Count.ToString(); } } |
Here the tested code relies on the MessageCenter.Count static property. Mocking can help here as well.
|
1 2 3 4 5 6 7 8 9 |
[TestMethod] public void GetCountFromCenter_ItemCountIsZero_NoNewMessages() { Mailbox mailbox = new Mailbox(); Isolate.WhenCalled(() => MessageCenter.Count).WillReturn(0); var result = mailbox.GetCountFromCenter(); Assert.AreEqual("No new messages.", result); } In this case, we use Isolator to specify the behavior of the Count property, in the same way, we do for objects. |
Tests that use mocking are fast since they replace slow behavior (calling the file system, calling the cloud) with in-memory operations. They are focused since they neutralize the dependency effects on the tested code, and therefore also make the tests deterministic.
Using mocks require coupling to the code. Readability of the tests, therefore, relies on how the code under test looks like. The more complex it is, the less readable the test is.
Summary
We have seen the basic patterns used in writing unit tests, all unit tests should follow these patterns. As the tests need to be:Fast, Robust, Readable, Focused, Deterministic and Independent, you will find that this will improve your code’s design.
- Great Tests – Effect on production code design
- Fast – Concise methods
- Robust – Design is opened to change
- Readable – Better APIs
- Focused – Each class and method will have a well-defined responsibility
- Deterministic – Design decouples code from environmental effects
- Independent – Design does not rely on shared or static state
In this article, we introduced the common unit-test patterns. These patterns can be used alone or in combinations. For example, you want to mock a database connection but it is created in a protected virtual method. The test will use the Inner Class Pattern to return the mock database object with a mock object for the actual database.
There are still many situations in which these unit-test patterns are not sufficient and there is a need to change the code to make it testable. These include:
- Singleton classes
- Calls to static members
- Objects that do not have an interface (as required for mock-object pattern)
- Objects that are instantiated in the code being tested
- Objects that are not passed to the method (as required for mock-object pattern).
Learn more: