The economics of testing

Having recently read "The undercover economist", an interesting peek behind the curtain at how economics influences our decisions (by the marvellous Tim Harford, whose radio series "More or less" is always worth a listen), it was obvious a lot of what he said was applicable to the field of testing, so armed with a list of basic economic principles (from I've had a go at framing them in a testing environment.

Because of Scarcity, People Choose: even in the most well-funded project there is only ever a finite amount of time available so choices have to be made - how we make those decisions and choose what to sacrifice is at the core of what a tester does; which parts of the product to focus on, deciding how much testing is "enough", and employing strategies such as touch testing all areas and using the information gained to revisit those areas of greatest concern.

All Choices Have an Opportunity Cost: one of the great testing skills is being able to realise when a decision is actually being made so that conscious reasoning is applied rather than treading the same well-worn path "just because". No two projects are the same; the best testers are adaptable and willing to try new paths as the situation dictates.

People Respond to Incentives in Predictable Ways: useful when trying to convince a project manager to follow a particular path. The best justification you can give for any test strategy is providing a tangible incentive; there's not a manager on earth who will turn down a suggestion that will increase test coverage and reduce costs, as long as you can back your argument up.

Market Forces and Economic Systems Influence Choices: in a nutshell, the funding stream for a project dictates the boundaries within which testing must operate. A company producing a single product over the long term will be more open to experimentation than, say, an agency where a short term failure will lead to immediate loss of business.

People’s Choices Have Intended and Unintended Consequences Which Lie in the Future: no testing strategy is perfect; despite what project managers may think, a bug that makes it through to live is not necessarily a failure of the testing, it may be that the cost of finding that issue may hugely outweigh the loss incurred from it being present - as long as it was a decision (albeit implicit) not to find that bug, the fact it was discovered in live can be used to inform the same decision the next time it arises.

People Gain When They Trade Voluntarily: aka make friends with your developers. Found bugs don't need to be seen as an indication of failure on the part of the coder, more a benefit of the testing process, and an opportunity for learning, therefore a benefit for everyone.

Exercises like this are invaluable in allowing you to refresh your approach - one of the easiest traps to fall into is stagnation, failing to remember why you do what you do. Taking a view from a different standpoint such as this forces you to re-evaluate and justify your actions: and that can only make you a better tester.