The word “test” has become quite a loaded term in the software world in recent times. What follows is a non-comprehensive list of the possible meanings you can ascribe to “test,” depending on who you are:
- A manual, script test.
- Very granular automated tests, written and run with the help of a dedicated test runner.
- Automated GUI tests, produced by recording a person interacting with a screen and then replaying it.
- Less granular automated tests, that integrate different modules and require a special context to be prepared for them to run on.
- Automated tests that populate the system with huge quantities of (possibly rotten) data to see how the application handles it.
You could also get a variety of different answers if you were to ask, “Who’s responsible for creating the tests?”
- The software developer who wrote the code.
- A software developer but not the same one who wrote the code.
- The manual tester assigned to the task by the developer who wrote the code.
- A dedicated test analyst.
- The QA engineer.
And the list could go on, but the important thing here is that you get the picture. “What is a test and who’s responsible for it?” is a question that can get a lot of drastically different answers depending on who’s asking, to whom they’re being asked, and under which circumstances they’re being asked.
To make things crystal clear: in this post, we’ll be referring to automated unit tests generally written by the same developer who wrote the code in order to ensure it meets their expectations by testing the smallest possible piece (a “unit”) of the application in isolation.
3 Properties of Quality Tests (and How to Recognize Them)
Now that we’re on the same page about the meaning of a “test,” it’s about time you get what you’ve come here for: the characteristics of a good test and how to identify and enable them.
Quality Tests Are Fast!
I struggled a little bit on deciding not only what should be the properties of a good test but also what their order should be. Among so many important qualities, I’ve elected speed as the most important. If you want your tests to be quality, make them fast. How fast, you ask? We’ll get to that in a minute. But first, let’s see why speed is so important.
Why Is Speed So Vital?
One of the magical moments of being a developer is when you get to the famous flow, also called the zone. It’s that different state of mind where you’re so deeply immersed in the task at hand that you don’t even perceive the time passing. The flow isn’t only hard to achieve: it’s harder to achieve again once you’ve been kicked out of it. (With this in mind, it isn’t so hard to understand why many software developers loathe meetings, open-space offices, and interruptions of any kind).
Now try to imagine how a developer feels when being constantly interrupted by a test suite that runs at a glacial pace? Not happy, to say the least. So, if the tests get in the developer’s way of getting things done, they just won’t run them as often as they should!
How to Recognize Fast Tests
How do you know your tests are fast enough? That’s actually a pretty easy question to answer: you measure them. Just run the tests and see how they do. For a reference, Visual Studio’s Test Explorer considers a test fast if it takes less than 100 ms to run. Your CI system is probably well equipped to provide you with reports and metrics about test runs among other things.
How to Write Fast Tests
OK, it seems that recognizing fast tests is easy enough. How do we go about writing them?
First of all, we must make sure that what we’re writing are truly unit tests and not integration tests in disguise. What do I mean by that? Take a look at Michael Feathers’ definition of a unit test:
A test is not a unit test if
- It talks to the database.
- It communicates across the network.
- It touches the file system.
- It can’t run at the same time as any of your other unit tests.
- You have to do special things to your environment (such as editing config files) to run it.
That’s one of my favorite definitions of a unit test and it’s peculiar in that it’s a negative definition: it lists a bunch of things that a unit test cannot do. One of the main things we get from this list is that a unit test can’t perform I/O at all. If it talks to the database, the file system, or any other external reality, then it’s not a true unit test.
And why not? Because messing with those things will slow your tests down. So the answer is pretty simple, actually: write tests that are truly unit tests.
Quality Tests Are Readable!
A great unit test is readable. Think about the benefits this brings to the table:
- A readable test will help the developer understand and troubleshoot the code faster when something goes wrong.
- Even though this vision might be somewhat controversial, it’s useful to think of unit tests as documentation or even executable specifications. Unreadable documentation is pretty much useless.
- A readable test makes it easier for the reader to understand the test itself, which can result in both a better understanding of the system under test and a reduced risk of damaging the test if the need to change it ever comes up.
How to Recognize Readable Tests
This one is unfortunately not as clear-cut as the previous point. When talking about readability, there’s no denying that there will be some degree of subjectivity. Despite that, I believe there’s an awesome tool you can use to gauge the readability of your tests: the code review.
Code review is a good practice in its own right, but it can be particularly effective when put to the service of guaranteeing quality in tests. Have your developers review each other’s tests, making a point that test readability should be a required criterion for the review to be approved.
How to Write Readable Tests
There are two main components of a readable test: its name and its content. Each one of them provides some unique value. A great test name can greatly enhance the efficacy of a developer to identify and fix an issue when a test run fails.
On the other hand, a test whose contents are soundly structured will help its readers to quickly understand it, while also providing insights on how the application works (executable documentation, remember?).
With this in mind, what should we do to achieve greater readability in our tests? First of all, let’s pick a good naming convention. There’s plenty of them for you to choose from, but I particularly like Rob Osherove’s convention, which follows the format [MethodName_StateUnderTest_ExpectedBehavior].
By employing this convention, we can quickly identify some very useful information, namely what’s being tested, under which circumstance, and what was supposed to happen.
With the name out of the way, let’s cover the content of the method. The advice here is two-fold:
- Use the Arrange-Act-Assert structure.
- Use sound variable names.
Quality Tests Are Independent!
Great unit tests should be independent. Not only from the external world but also from other tests. You should be able to run all the tests, some of the tests, just one of the tests, in any combination and order, and the result should always be the same.
How do you detect the property of independence in a test? If the test is fast, it’s likely to be independent. But how can you know for sure? By employing a test runner that allows the tests to be run randomly and concurrently. If the tests are truly independent, then they should run normally, despite the execution order and despite being executed simultaneously to other tests in the suite.
And how to go about writing them? The same way you write fast tests: by writing tests that are truly unit tests.
The Road to Quality Can Be Long, but It’s a Road Worth Traveling!
The journey to quality tests (or to quality, period) is often a tough one. But the destination—and the path itself—is rewarding and fulfilling. So put in the hard work: read, study, and practice a lot, use the right tools, and take your work to the next level!
This post was written by Carlos Schults. Carlos is a .NET software developer with experience in both desktop and web development, and he’s now trying his hand at mobile. He has a passion for writing clean and concise code, and he’s interested in practices that help you improve app health, such as code review, automated testing, and continuous build.
