I like to use mind maps to help me test. Mind maps are a visual way of brainstorming different ideas. On a previous mobile team I printed and laminated this mind map to bring along to every planning session to help remind me to ask questions like, “What about accessibility? Automation? Security? or Performance?”:
As I go through exploratory testing (or pair testing), I’ll tick things off as I go and take session notes. Often this will involve having conversations with people, sometimes bugs are raised. Here is a quick mind map I’ll use for log in testing:
Heuristics for testing
This mind map approach can be combined with a heuristic test strategy approach or a nemonic test approach. Heuristics is a rule of thumb that helps you solve problems, they often have gaps because no mental model is perfect.
SFDPOT is a common nemonic that was developed by James Bach; who also developed Rapid Software Testing; a context driven methodology. James Developed his RST course with Michael Bolton.
Have you wanted to start with automation testing and not known where to begin? Or maybe you have 100’s or thousands of test cases in your current automation pipeline and you want to reduce the build times. Here I will walk you through one way you could consider slicing up this problem. Using examples from Tyro’s banking app (I use to work on their mobile iOS team).
Break into flows
Analyse your app/site/tool and brainstorm the main flows that people will take through it. I picked 6 flows using tyro as an example app. Next I numbered them.
2. Transfer Funds
3. View Transaction
4. Contact Us
5. Change Pin
6. Log in
Mapping those flows to a risk board
Draw a graph, put frequency of use on the x axis down the bottom; things that are more used will be on the right hand side. On the vertical y axis put impact if broken. This is from a person point of view, how much would they care if that feature was a broken? From a business point of view you may have a different understanding of risk and that’s fine two. We will go into how to reflect that later.
Add your flows
Move the flows to your graph
It helps to pair on this exercise to help build up a shared understand. Do your designers and engineers have the same understanding of risk as you do? It’s ok if your answer is different to mine, we all have a different context and understanding.
Reflect other elements of risk
You might want to reflect other elements of risk such as security, financial, regulatory and anything else you can think of. At the end of the day this is only a 2 representation of risk and risk is a little more complex than these dimensions we put here.
Neat, what’s next?
If you are thinking, well that’s cool and all but what does that have to do with automation testing? Then please continue reading. You could use this board to decide which tests you should focus on building/refactoring next (hint, the stuff with 3 stars is pretty important). You could also use this to priortise your performance testing efforts. I took this board to our planning sessions to talk about new features and it helped with deciding how much automation/testing effort we may need. At the end of the day, your software will be more complex than this example.
Here is the actual board I used at Tyro with a bit more detail:
I then broke down each flow into a test case, and grouped similar test cases into a barebones automation test suite. You can also use this approach to generate exploratory testing ideas for each screen in your flow.
Soap Opera Testing is a dramatised method used for testing your business processes. You might want to try it for a super-condensed and thorough way of highlighting bugs. And because it’s fun. Embrace the drama.
Cem Kaner has been writting about scenario testing for a long time. He published this article on ‘an intro to scenario testing’ and Hans Buwalda presented on ‘soap opera testing’ nearly 20 years ago 😱. They’re both serious tester dudes and this stuff is legit.
How Does it Work?
You might start with a brain storming session with your sales or customer support team. Ask them for stories about things your users have done. Not just the ordinary things, but also some off-the-wall and crazy things. What you’re looking for is drama.
It might help to sketch out the story briefly. Write down steps that are essential, or those that you might make a mistake on. Cem Kaner gives some practical tips here, although you definitely don’t need to read all 500 pages.
Kaner’s Introduction to Scenario Testing is a bit more bite sized and describes the five main points your scenario needs. Namely that it’s a story, it’s credible, it will test the program in a complex way, the results are easy to evaluate and stakeholders will see the point of fixing the bugs identified.
A scenario is a hypothetical story, used to help a person think through a complex problem or system.
You then run a test exercise using the characters and scenarios from a soap opera, and analyse the results. You can do this as many times as you want, with as many different scenarios.
Use whatever soap opera you like. We make no judgements. Although fair to say that if you use A Country Practice, you’re showing your age and nobody will know what you’re talking about.
Let’s Soap Up
Here’s an example of Soap Opera Testing using The Simpsons. The program being tested is a mortgage loan application.
Let’s say Homer Simpson wins the lottery, and decides to apply for a second mortgage, for an investment property. Just as the paperwork is about to go through, Grampa Simpson burns his apartment down.
Homer decides to help him out with the cost of a rental, meaning he needs to change the deposit he’ll pay on his investment property. Homer signs the amended paperwork but he signs it incorrectly.
Then his application is declined because even winning the lottery doesn’t give you a good credit rating overnight. The Simpsons’ next ‘diddly-door’ neighbour Ned Flanders offers to help Homer out. He’ll put in the 10% deposit.
Ned lends a helping hand
His own house is 90% paid off so it’s no big deal to him, and it will help Homer get around his bad credit rating. The Simpsons’ house is 50% paid off, and they’re putting down a 90% deposit, using Homer’s lottery winnings, and leaving some bowling money left over.
They’re about to go to the bank and lodge the paperwork, when Homer’s half-brother Herbert Powell hears about the lottery win. Boy has he got the mother of all investment options for Homer – nuclear powered cars!
Adjust down payment
Homer can get in on the action if he puts some cash into building a prototype. So Homer has to syphon off yet more funds from the deposit he’ll make on his investment property, and change the paperwork again.
Whew, put all that through the system and see where you get to. If you think of more variables as you go, you can add them to the scenario and run the test again.
What We’ve Tested
A whole heap of stuff.
We tested rejections, with Homer’s first application, and signature recognition when he goofed up his name.
We tested multiple applications made by the same person, with an adjustment in the deposit amount made after the application had gone through.
We tested how to register multiple assets with different mortgage amounts, and a different percentage of ownership. What’s more, the owners of the properties and mortgage were not residents at the same address.
The applicants had different credit ratings, which affected the different algorithms in their application process. And they weren’t related, and didn’t intend living together at the property, which was for investment only.
Here’s a snappy list:
Multiple applications from the same person
Multiple assets with different mortgages
Different percentage ownership
Different credit ratings
Investment property applications
It only takes a little imagination to try to find many more bugs using a soap opera scenario, versus the standard “works as expected” response we’d have gotten from the test-case walk through.
Here’s a three minute recap in a lightning presentation I gave at the Selenium Conference in India.
You should watch this TED talk by Hans back in 2007:
The 10 Dramatic Instances
Hans goes through 10 biases that cause us to over dramatise the world we live in. he kicks off the book with a questionnaire that helps highlight the incorrect world view we all hold in our heads. Alot of these instinct feed into our own internal fears and make them even louder. E.G. the fear of a rapidly changing tech scene that we can’t keep up with.
Population growth will not explode
It is already starting to even out. The upper estimate now is around 11 billion people by 2100. I really like this visualisation by Hans: