This is the latest in a series of posts about unit testing in SharePoint that The SharePoint Baker has written. For his previous posts, see Parts 1,2,3, 4.
I thought I’d share something of my experience so far with unit testing SharePoint using Typemock. For a bit of background, I’ve been developing with SharePoint for the last couple of years and have been involved with a number of fairly large scaled, highly customised solutions which have contained a lot of custom code. Up until recently, all of the testing that was carried out was what I guess you’d call integration testing – testing was done by users in the browser actually using the functionality. This testing was done both by individual developers and dedicated testers who would generate their own set of test scripts. Those testers would summarise the problems and then pass them over to the developers to then fix. This approach to testing is fairly common and typically its the easier way to test (I’m not suggesting the tests are easier, it’s just they normally don’t require any extra software as the testing is carried like an end-user would use the system). One of the downsides to testing is the overhead that comes from finding bugs, understanding them, reporting, finding in code, fixing, local testing, new build, deploy, re-test! There’s the idea that the earlier a bug is found, the cheaper it is to fix and that is absolutely right. From experience I really believe that unit tests can make a difference- especially from a developers point of view. Having a series of tests that can be run against changes during the build process goes a long way to finding problems earlier. If I know that the latest build has issues because its failed some unit tests I won’t even release it to the testing team to start their testing – instead I can address those issues and produce a new release which hopefully means less bugs for them to find. I won’t necessarily be able to remove every bug but I can certainly help reduce the number with unit tests. If I can reduce, even by half, the number of bugs the testers find it will have reduced the amount of rework needed – of course really we’d be looking at a higher number!
Performing any sort of SharePoint unit tests is a nightmare because the majority of the SharePoint object model is sealed and relies on a SharePoint site being available (Consider SPSite whose constructor needs a site Url or Id). This is where Typemock comes in because it gives us an isolation framework which we, as developers, can use to create unit tests without the overhead of having SharePoint there. So over the last few months I’ve done two main types of testing with Typemock for SharePoint- the first has been writing tests alongside new development. I’ve not adopted a test driven development (TDD) approach to the tests but have just written the tests and code in parallel. And I have found that this has been beneficial in a number of ways. Firstly is having to constantly think about the tests actually helped to shape the code that was being written. The question of “how this could be tested” helped to shape it – especially where error trapping and defensive coding came into play. For example, one of the typical test scripts would be to try and pass in a null and string.Empty value into a method (e.g. MyRandomMethod(null)) and so defensively coding against these has now become almost second nature because of the number of times I’ve had to write that type of unit test. Because of how Typemock works once a test script has been written its easy enough to replicate it and to pass in different values to be tested meaning that its very easy to test lots of different scenarios with different data. The second type of testing that I’ve done has been to actually start writing test scripts for existing code. This has been a bit more challenging because the structure of the code hasn’t necessarily lent itself to testing and whereas during development that code could be refactored this isn’t always an option with existing code. The test scripts that I’ve written so far have all highlighted improvements that could be made and a number of areas of refactoring. Refactoring production code is always interesting because whilst there are usually definite benefits to doing the refactoring this is always weighed against introducing new unforeseen bugs into the code. What is good is that when this refactoring is done these unit tests can be run again against the refactored code and the result should be the same and as a developer we can more confident in those changes.
I guess the question is have these unit tests and Typemock made a difference in real projects? So far it has. The bits of new development that I’ve been able to write tests scripts for have benefited from the unit tests – the number of bugs and issues found during integration testing were noticeably fewer than previous pieces of development. Where bugs were found its help to refine those tests scripts to improve their coverage. The benefit for test scripts for existing code is less obvious – but that is more to do with the fact that the code can’t always be refactored “on the fly”. So will using Typemock ensure code is perfect? No, but then no approach to testing is guaranteed to produce perfect code. Typemock makes it fairly easy to write a test scripts and once you’ve done a few tests its straight forward to roll out more scripts. Like any new piece of software there’s always something of a learning curve but Typemock provide great support and there’s a number of online resources that provide examples. I think it’s important to remember that Typemock provides the ability to do testing and not the tests themselves – we’ve still got to write those! We’ve got to be careful that we don’t just focus on getting get tests to pass, we’ve got to make sure those tests are actually testing the code. We can’t just write tests that pass because ultimately we do unit testing to improve quality and not just to tick a box to say we’ve done testing. It’s sometimes too easy to look at the easier scenarios with the right type of information rather than the wrong information. If we take the same approach with unit tests – only testing the “simple” things or the things we know are right, then they’re not really worth doing. To take a simple example, say we have a form that captures a user’s name and there’s some validation on this field, checking its length is between a minimum and maximum value, only allows certain characters etc. One unit test could simply be checking a valid value. That was easily done, unit test written! Of course in reality that’s not the case – weak unit tests won’t help and won’t improve quality. We should have tests that check for invalid values, too long, too short, invalid characters etc. If these “simple” test are the types of tests that are being produced I’d argue that they’re not worth writing to begin with. As a developer you wouldn’t write a method that didn’t do anything or one that was never going to be used and the same thinking can be applied to unit tests. Of course the flip side of this argument is how many scripts should be written? Writing and maintaining unit tests take time and sometimes there are commercial decisions which means we can’t spend the time scripting every scenario.
If you’re serious about unit testing bespoke SharePoint development using a framework like Typemock I think is essential. Without it there’s so much code that just can’t be tested with any confidence. There’ll never be 100% code coverage with SharePoint testing and there are areas that Typemock can’t be used to test completely but it goes a long way and my view is that some coverage is better than none! I hope my musings have made some of sense and if you’d like any info or more details of what I’ve been looking at feel free to leave me a comment. If you’re interested, I’ve got a couple of previous posts giving more code based examples of unit tests with Typemock and you can find a list of those here:
Happy testing!
