That elusive Test Strategy

I recently was asked about recommendations for learning about test strategies. Here are my sample strategies:

• A Mobile app Test Strategy
• A Marketing Strategy

a strategy doesn’t have to be a big giant document. It starts as an idea in your head and you have to get other people on board as part of that strategy. So you need to share some knowledge in some format to help share your idea. This blog is about how I’d go about developing a new test strategy in a new team.

History of the term

First let’s take some time to understand this term; strategy. Historically the word strategy is associated with war and battle:

Strategy is to help you win or achieve some goal. Many people talk about their tactics when they are thinking of their strategy. Tactics are your how. They aren’t your whole strategy.

A tactic is a conceptual action or short series of actions with the aim of achieving of a short-term goal. This action can be implemented as one or more specific tasks.

https://en.wikipedia.org/wiki/Tactic_(method)

Book: I have a strategy (No you don’t)

This book helped me understand the term, “Strategy” in a visual and fun way.

According to this book a strategy has 4 parts:

• A purpose
• A distinct, measurable goal
• A plan
• A sequence of actions or tactics

Start with a purpose

If I was dumped into a new team tomorrow and asked to develop a test strategy, I’d start by interviewing/surveying a few people. Depending on the size of the team and who I was working with it could be an online survey or a casual chat over a coffee. I’d ask something along the following lines:

• What does quality mean to you?
• What are common problems in the testing process here?
• If you could fix just one thing about our quality, what would it be?

Now different people are going to answer this differently. Developers might say test code coverage, easily maintainable code and easy deployments make a high quality product. Your project manager might say happy customers. Testers might say less bugs found in the test phase.

Develop a goal

Once I’ve surveyed enough people (5 people is a good enough number for most user research interviews), I’ll work on constructing a goal. it might be;

• improve our continuous integration build times
• increase our test coverage
• reduce the amount of negative customer feedback

Make sure it is measurable. You could use SMART or OKR goal formats.

Develop a plan

Now what are some things I or the team could do to achieve our goals? We could create tasks during our sprint to help us work towards our goal. Once you’ve achieved something you survey those original interviewers to see if the perceived quality has actually improved.

Measure your progress

Measure the improvements in quality of your product. For my team we are tracking the average app store ratings, crash rates and engagement with in app features to see if they are actually useful. https://bughuntersam.com/metrics-and-quality/

Risks and Gaps

A Test Strategy could also have a section about risks or gaps in this approach. For example things like performance testing and security testing might not be included. Having a brief explainer why these aren’t part of your strategy can be useful for explaining the context and scope.

UI Automation Visual Risk Framework

if you are working on improving the UI Test automation coverage you can use this visual risk based framework to help focus on where to start and what to automate first and measure progress against it as part of your strategy.

Conclusion

I’m more comfortable with the term marketing strategy over test strategy because it’s easier to measure your impact and easier to come up with concrete goals. Software testing isn’t as tangible as many other parts of the business process and can be hard to measure.

Can your strategy be summarised by this comic:

What resources have helped you understand test strategies? I’d love to check them out.