Before each release the support guys click through the whole application (following a smoke test script) to prevent that new bugs sneak in. However, sometimes they don’t manage to find all of them. No wonder since humans just tend to make mistakes when doing such stupefying, reoccuring work. Apart from that, the procedure is awfully time consuming – eight days of development require two days of testing.
That’s why we decided to enhance our suite of acceptance tests. Now we use exactly those two days support needs for testing to write new Fitnesse tests. We first created a list of all parts of the smoke test which were not already covered by automated tests. Each pair of developers picks one of the tests, implements it and then the interaction design girls look over it to ensure its quality. That way we already implemented 2/3 of the missing tests. As a side effect we improve old Fitnesse tests we find while searching for “inspiration” for the new ones.
Because Fitnesse tests are run on our continuous build server after each commit, new bugs are found before the smoke test even starts.
And that pays off: In the last three iterations only one small mistake made it into the release! Customers and support love our improved Fitnesse.