Coming back from Agile Testing Days 2013 (you can read my personal impressions here), I thought this would be a good opportunity to talk about cold, triangly-shaped things.
Exhibit A is from Andrea Tomasini’s keynote “Agile Testing is nonsense, because Agile is about testing!”. It covered a lot of topics, but on slide 19, you’ll see this:
In this slide Andrea talked about the Ice Cream anti-pattern (left-side), which is counter to Mike Cohn’s testing pyramid (right-side).
Mike Cohn described the kinds of (automated) tests that should in your arsenal. Basically you should have all kinds of tests: automated unit, integration and acceptance tests, as well as ad-hoc, or manual exploratory tests. You should have ALL kinds, the important thing is how many you have of each.
The pyramid is shaped this way, because most of your tests (and therefore effort in writing and maintaining) should be quick running unit tests. They are easy to write, fast to run, pinpoint problems when they fail, and fairly easy to fix or remove when functionality changes.
The more you climb up the pyramid, tests become more expensive, and the feedback you get from them slows down. If the tester takes two days to test a feature, that is costly. If you could replace most of what he’s testing with automated tests, it won’t takes two days to get the results back , and you’ll free him to do some exploratory testing.
Unfortunately, the ice-cream anti-pattern is very common: most testing is manual. Companies pour lots of money into the QA department, and not enough into writing automated tests (which can save them money later). When actual automation takes place, the automation effort is usually complex, because the tests run the application from the UI inwards. Feedback is slow, the tests are fragile, and so, in time, we don’t see much value in them. We’re working for the tests, where it should be the other way around.
Interestingly enough, another triangle popped-up in Seb Rose’s talk “Don’t you trust me?” about BDD (behavior-driven development) and creating trust between business people, developers and testers. Seb talked about automated user acceptance tests, and their usefulness. Coming to this topic from another angle, tilting the pyramid and keeping the cool metaphor (pun intended), to create an iceberg (slide 36):
Just like in Mike Cohn’s pyramid, in Seb’s iceberg there are different types of test, but he adds distinction between business- and developer-facing tests. The business-readable tests (e.g. not code, that only developers can understand, sometimes) are the end-to-end ones (similar to the user acceptance tests), and from the point of view of the business people, they are the majority. The integration tests somehow split between business-readable and code. Note that some of the unit tests (not the majority), should also be business readable. For example, what error message the user sees on the screen, can be specified as a user-acceptance test, but runs quickly, just like a regular unit test. Of course, the majority of the tests are still unit tests, and most of them are developer tests (that only developers can understand. Sometimes.)
I like how these perspectives come together, because they complement each-other. Mike Cohn’s pyramid is based on value and cost: what kind of feedback am I going to get, how long will it take, and how much will it cost me. Seb splits the tests a bit differently, by adding trust into the equation: the user-acceptance tests foster communication between the developers and business people.
You can always go for the ice-cream, but I bet it won’t be as good.
Like his writing? Watch Gil’s webinar: how to fix a bug.
