Modern testing principles

I heard a podcast discussion recently on a really interesting attempt by AB testing to define a set of Modern Testing Principles in a similar way to the Agile Manifesto. There's a lot of thought-provoking stuff in there around why we test and what our goals as testers should be; it certainly made me think more about the role of the test team within a project. Here I've listed the the Mission Statement and 7 Principles as stated at time of writing and included some of my thoughts. First the mission statement:

Accelerate the Achievement of Shippable Quality

This assumes the end goal is shippable quality, i.e. the tester's role is to ensure things are delivered fully functioning (relative to the specification), and the aim is to make that process as fast as possible (not working harder/faster, but smarter, and aligning processes to match that goal). There may be cases where this doesn't apply - "shippable" can potentially be interpreted in a few ways and you may wish purposely release a product that is not finished, but as a general statement this gives a good framework to discuss the individual principles.

1. Our priority is improving the business.

Absolutely. I'm paid to do a job; the overall focus of that job has to be to make money for the company so they can continue to pay me. That doesn't mean doing things on the cheap - short cuts may give short-term benefits but are likely to cause long-term damage.

2. We accelerate the team, and use models like Lean Thinking and the Theory of Constraints to help identify, prioritize and mitigate bottlenecks from the system.

Makes sense - regardless of who you are, everything you do should be contributing to the efficiency of the delivery process (although I would be tempted to leave out the references to specific models to avoid this going out of date).

3. We are a force for continuous improvement, helping the team adapt and optimize in order to succeed, rather than providing a safety net to catch failures.

I see this as an attempt to stop the "throw it over the wall" developer mindset. The old model of develop/test/redevelop/retest is not efficient. There is a spectrum of development styles ranging from developers attempting to deliver perfect code (in which case testing time is redundant as it has a low return); mid-line developers throwing stuff at you knowing it doesn't work; or in the extreme, raising of bugs before code is written (TDD). Each approach will take different amounts of time; we need to find the one that leads to the shortest overall project time, taking into account constraints such as timelines, available staffing levels and the end goals of the project (assuming these are non-negotiable, at least in the short term). So the focus is on total project time, not test time or dev time. Test could make their own time shorter but that might push the time onto the devs and not benefit the project overall (and vice versa); we need to take a full project view rather than just minimising time spent in a particular discipline.

4. We care deeply about the quality culture of our team, and we coach, lead, and nurture the team towards a more mature quality culture.

Testers have to work as a team in order to increase efficiency. This includes the ability to swap between testers during a project (i.e. making sure we all work to the same methodology, or at least have ways of making it clear which methodology we are using) so that short-term resourcing issues can be overcome without major impact on a project's timelines.

5. We believe that the customer is the only one capable to judge and evaluate the quality of our product

This depends on the definition of customer. Following on from Principle 1, I think this has to be the person that is supplying the finance behind the project. While we are able to suggest amendments that might be beneficial to the end-user, we must not lose sight of the needs of the person paying for the work, who has to be the ultimate arbiter of what is good for the product; we can obviously make suggestions based on our knowledge and experience, but the customer may have a separate agenda and therefore should make the final choice.

6. We use data extensively to deeply understand customer usage and then close the gaps between product hypotheses and business impact.

This feeds in to Principle 5 - the assumption has to be that the customer is trying to influence the business impact in some way, and we have to be able to back up any decisions we make to show that they will actually have (or be likely to have) that impact.

7. We expand testing abilities and knowhow across the team; understanding that this may reduce (or eliminate) the need for a dedicated testing specialist.

To me this is saying that testing is not a ringfenced discipline. We are not here to ensure work is created for testers, but to make sure the right thing is done for the project, even if that results in a reduction in the amount of testing carried out.

In summary, it's all about doing the best for the project, not for the tester (or the test team). Going back a few years, the focus was often on making the test team more visible and encouraging teams to understand why testing was important - I think that message has been taken on board now (mostly) and the presence of the testers is a given in most projects. These principles reflect a consolidation of that position by ensuring testing activities are necessary and relevant; if we prove our worth (even by recognising areas where test does not need to get involved) the continuing presence of the test team within projects will be ensured.