I like to use mind maps to help me test. Mind maps are a visual way of brainstorming different ideas. On a previous mobile team I printed and laminated this mind map to bring along to every planning session to help remind me to ask questions like, “What about accessibility? Automation? Security? or Performance?”:
As I go through exploratory testing (or pair testing), I’ll tick things off as I go and take session notes. Often this will involve having conversations with people, sometimes bugs are raised. Here is a quick mind map I’ll use for log in testing:
Heuristics for testing
This mind map approach can be combined with a heuristic test strategy approach or a nemonic test approach. Heuristics is a rule of thumb that helps you solve problems, they often have gaps because no mental model is perfect.
SFDPOT is a common nemonic that was developed by James Bach; who also developed Rapid Software Testing; a context driven methodology. James Developed his RST course with Michael Bolton.
In SFDPOT each letter stands for something that is meant to help generate testing ideas, they are; Structure, Functions, Data, Platform, Operations and Time. Here’s someone’s blog on applying SFDPOT to a mobile app testing approach. I tend to use all of these different approaches as part of my exploratory testing practices.
Your team could also go through the process of creating your own nemonic/mindmap to help you have consistency across different testing styles, maybe something that made sense for your context?
You could also use mindmaps to feed into the different feedback loops you’d like to have in your team. You can read more in this mobile test strategy blog.
Have you wanted to start with automation testing and not known where to begin? Or maybe you have 100’s or thousands of test cases in your current automation pipeline and you want to reduce the build times. Here I will walk you through one way you could consider slicing up this problem. Using examples from Tyro’s banking app (I use to work on their mobile iOS team).
Break into flows
Analyse your app/site/tool and brainstorm the main flows that people will take through it. I picked 6 flows using tyro as an example app. Next I numbered them.
1. Registration
Registration is a pretty common feature, you might also set a 2 factor authentication, a pin and a password for the account (especially if it’s a bank account)
2. Transfer Funds
If you have a bank account, it’s highly likely you want to access the money in it at some point
3. View Transaction
You might want to check if that bill was paid correctly or if the last transfer was processed
4. Contact Us
Something not quite right? send us a request and we will give you a phone call at a convenient time
5. Change Pin
When was the last time you changed the pin for your mobile banking app?
6. Log in
I’d say this is a pretty common feature
Mapping those flows to a risk board
Draw a graph, put frequency of use on the x axis down the bottom; things that are more used will be on the right hand side. On the vertical y axis put impact if broken. This is from a person point of view, how much would they care if that feature was a broken? From a business point of view you may have a different understanding of risk and that’s fine two. We will go into how to reflect that later.
Add your flows
We have our 6 flows to the right hand side of our graph, we’ve also broken our graph into 3 areas
Move the flows to your graph
It helps to pair on this exercise to help build up a shared understand. Do your designers and engineers have the same understanding of risk as you do? It’s ok if your answer is different to mine, we all have a different context and understanding.
Reflect other elements of risk
You might want to reflect other elements of risk such as security, financial, regulatory and anything else you can think of. At the end of the day this is only a 2 representation of risk and risk is a little more complex than these dimensions we put here.
Neat, what’s next?
If you are thinking, well that’s cool and all but what does that have to do with automation testing? Then please continue reading. You could use this board to decide which tests you should focus on building/refactoring next (hint, the stuff with 3 stars is pretty important). You could also use this to priortise your performance testing efforts. I took this board to our planning sessions to talk about new features and it helped with deciding how much automation/testing effort we may need. At the end of the day, your software will be more complex than this example.
Here is the actual board I used at Tyro with a bit more detail:
I then broke down each flow into a test case, and grouped similar test cases into a barebones automation test suite. You can also use this approach to generate exploratory testing ideas for each screen in your flow.
You can watch this talk in full here:
I also run this as a lunchtime 30-45 minute workshop exercise. Book me in for a lunchtime brownbag if you are based in Sydney (I can do remote too).
Soap Opera Testing is a dramatised method used for testing your business processes. You might want to try it for a super-condensed and thorough way of highlighting bugs. And because it’s fun. Embrace the drama.
Origins
Cem Kaner has been writting about scenario testing for a long time. He published this article on ‘an intro to scenario testing’ and Hans Buwalda presented on ‘soap opera testing’ nearly 20 years ago 😱. They’re both serious tester dudes and this stuff is legit.
How Does it Work?
You might start with a brain storming session with your sales or customer support team. Ask them for stories about things your users have done. Not just the ordinary things, but also some off-the-wall and crazy things. What you’re looking for is drama.
It might help to sketch out the story briefly. Write down steps that are essential, or those that you might make a mistake on. Cem Kaner gives some practical tips here, although you definitely don’t need to read all 500 pages.
Kaner’s Introduction to Scenario Testing is a bit more bite sized and describes the five main points your scenario needs. Namely that it’s a story, it’s credible, it will test the program in a complex way, the results are easy to evaluate and stakeholders will see the point of fixing the bugs identified.
A scenario is a hypothetical story, used to help a person think through a complex problem or system.
You then run a test exercise using the characters and scenarios from a soap opera, and analyse the results. You can do this as many times as you want, with as many different scenarios.
Use whatever soap opera you like. We make no judgements. Although fair to say that if you use A Country Practice, you’re showing your age and nobody will know what you’re talking about.
Let’s Soap Up
Here’s an example of Soap Opera Testing using The Simpsons. The program being tested is a mortgage loan application.
Let’s say Homer Simpson wins the lottery, and decides to apply for a second mortgage, for an investment property. Just as the paperwork is about to go through, Grampa Simpson burns his apartment down.
Investment property
Homer decides to help him out with the cost of a rental, meaning he needs to change the deposit he’ll pay on his investment property. Homer signs the amended paperwork but he signs it incorrectly.
Then his application is declined because even winning the lottery doesn’t give you a good credit rating overnight. The Simpsons’ next ‘diddly-door’ neighbour Ned Flanders offers to help Homer out. He’ll put in the 10% deposit.
Ned lends a helping hand
His own house is 90% paid off so it’s no big deal to him, and it will help Homer get around his bad credit rating. The Simpsons’ house is 50% paid off, and they’re putting down a 90% deposit, using Homer’s lottery winnings, and leaving some bowling money left over.
Side investment
They’re about to go to the bank and lodge the paperwork, when Homer’s half-brother Herbert Powell hears about the lottery win. Boy has he got the mother of all investment options for Homer – nuclear powered cars!
Adjust down payment
Homer can get in on the action if he puts some cash into building a prototype. So Homer has to syphon off yet more funds from the deposit he’ll make on his investment property, and change the paperwork again.
Whew, put all that through the system and see where you get to. If you think of more variables as you go, you can add them to the scenario and run the test again.
What We’ve Tested
A whole heap of stuff.
We tested rejections, with Homer’s first application, and signature recognition when he goofed up his name.
We tested multiple applications made by the same person, with an adjustment in the deposit amount made after the application had gone through.
We tested how to register multiple assets with different mortgage amounts, and a different percentage of ownership. What’s more, the owners of the properties and mortgage were not residents at the same address.
The applicants had different credit ratings, which affected the different algorithms in their application process. And they weren’t related, and didn’t intend living together at the property, which was for investment only.
Here’s a snappy list:
Rejections
Editing documents
Multiple applications from the same person
Adjusting deposits
Multiple assets with different mortgages
Different percentage ownership
Different credit ratings
Unrelated co-owners
Investment property applications
It only takes a little imagination to try to find many more bugs using a soap opera scenario, versus the standard “works as expected” response we’d have gotten from the test-case walk through.
Here’s a three minute recap in a lightning presentation I gave at the Selenium Conference in India.
I finished reading Factfulness by Hans Rosling on my commute this morning. Even Bill Gates recommends giving this book a read. Hans goes over 10 biases that cause people to feel like to world is much worse than what it actually is. The book is easy to read and I like the stories of Hans questioning his own biases. There is also this 4 minute read summary of the book too.
You should watch this TED talk by Hans back in 2007:
The book is an extended version of this amazing stats talk
Hans goes through 10 biases that cause us to over dramatise the world we live in. he kicks off the book with a questionnaire that helps highlight the incorrect world view we all hold in our heads. Alot of these instinct feed into our own internal fears and make them even louder. E.G. the fear of a rapidly changing tech scene that we can’t keep up with.
Population growth will not explode
It is already starting to even out. The upper estimate now is around 11 billion people by 2100. I really like this visualisation by Hans:
