Soap Opera Testing is a dramatised method used for testing your business processes. You might want to try it for a super-condensed and thorough way of highlighting bugs. And because it’s fun. Embrace the drama.
Cem Kaner has been writting about scenario testing for a long time. He published this article on ‘an intro to scenario testing’ and Hans Buwalda presented on ‘soap opera testing’ nearly 20 years ago 😱. They’re both serious tester dudes and this stuff is legit.
How Does it Work?
You might start with a brain storming session with your sales or customer support team. Ask them for stories about things your users have done. Not just the ordinary things, but also some off-the-wall and crazy things. What you’re looking for is drama.
It might help to sketch out the story briefly. Write down steps that are essential, or those that you might make a mistake on. Cem Kaner gives some practical tips here, although you definitely don’t need to read all 500 pages.
Kaner’s Introduction to Scenario Testing is a bit more bite sized and describes the five main points your scenario needs. Namely that it’s a story, it’s credible, it will test the program in a complex way, the results are easy to evaluate and stakeholders will see the point of fixing the bugs identified.
A scenario is a hypothetical story, used to help a person think through a complex problem or system.
You then run a test exercise using the characters and scenarios from a soap opera, and analyse the results. You can do this as many times as you want, with as many different scenarios.
Use whatever soap opera you like. We make no judgements. Although fair to say that if you use A Country Practice, you’re showing your age and nobody will know what you’re talking about.
Let’s Soap Up
Here’s an example of Soap Opera Testing using The Simpsons. The program being tested is a mortgage loan application.
Let’s say Homer Simpson wins the lottery, and decides to apply for a second mortgage, for an investment property. Just as the paperwork is about to go through, Grampa Simpson burns his apartment down.
Homer decides to help him out with the cost of a rental, meaning he needs to change the deposit he’ll pay on his investment property. Homer signs the amended paperwork but he signs it incorrectly.
Then his application is declined because even winning the lottery doesn’t give you a good credit rating overnight. The Simpsons’ next ‘diddly-door’ neighbour Ned Flanders offers to help Homer out. He’ll put in the 10% deposit.
Ned lends a helping hand
His own house is 90% paid off so it’s no big deal to him, and it will help Homer get around his bad credit rating. The Simpsons’ house is 50% paid off, and they’re putting down a 90% deposit, using Homer’s lottery winnings, and leaving some bowling money left over.
They’re about to go to the bank and lodge the paperwork, when Homer’s half-brother Herbert Powell hears about the lottery win. Boy has he got the mother of all investment options for Homer – nuclear powered cars!
Adjust down payment
Homer can get in on the action if he puts some cash into building a prototype. So Homer has to syphon off yet more funds from the deposit he’ll make on his investment property, and change the paperwork again.
Whew, put all that through the system and see where you get to. If you think of more variables as you go, you can add them to the scenario and run the test again.
What We’ve Tested
A whole heap of stuff.
We tested rejections, with Homer’s first application, and signature recognition when he goofed up his name.
We tested multiple applications made by the same person, with an adjustment in the deposit amount made after the application had gone through.
We tested how to register multiple assets with different mortgage amounts, and a different percentage of ownership. What’s more, the owners of the properties and mortgage were not residents at the same address.
The applicants had different credit ratings, which affected the different algorithms in their application process. And they weren’t related, and didn’t intend living together at the property, which was for investment only.
Here’s a snappy list:
Multiple applications from the same person
Multiple assets with different mortgages
Different percentage ownership
Different credit ratings
Investment property applications
It only takes a little imagination to try to find many more bugs using a soap opera scenario, versus the standard “works as expected” response we’d have gotten from the test-case walk through.
Here’s a three minute recap in a lightning presentation I gave at the Selenium Conference in India.
2018 has come and gone. You know what, overall it’s been a pretty shit year for me but the goal of this blog post is to take a moment to practice gratefulness and to reflect.
The crappy bits
I started 2018 not being able to walk because I broke my ankle in December 2017 and couldn’t walk for 12 weeks. I was also starting a new job. 2018 saw me go through a relapse of depression because of broken leg blues and I went through 2 jobs. The first one wasn’t the right fit and the second one had cash flow problems. But still, these were unplanned events that made me feel like failure.
However that is enough moping about. What else did I achieve in 2018?
I spoke at the Selenium Conf in India in July. This was pretty cool. It doesn’t matter how much people tell you about cultural differences, it’s really worth experiencing some of these things yourself. I had never been to India before so that was exciting.
I spoke at a few more in Aus (Agile Australia, and Australian Testing days), you can see all of my recorded presentations here. I also attended a few offering my sketch noting services. I enjoyed being able to attend these conferences and add some value back.
Getting around on wheels made me appreciate public transport here in Sydney. Sure it’s not 100% perfect, but it’s definitely better than New York. Watch Zach Anner on his quest for the rainbow bagel using the New York public transport:
I had a blast at Australian Testing Days Conference in Sydney on Friday. It’s always good to get reconnected with some colleagues and to make new connections. First up we had Jennie Naylor go over how to use OKR’s to drive the Quality Onwership:
The OKR (Objectives and Key Results) framework has been in use since the 70’s and is used by companies such as Google, Uber and Facebook to set their organisational goals. OKR is a framework that can be used at any level within your organisation. This presentation mention how to use OKRs to help improve quality and ownership in a team.
The key takeways I got from this talk was how to use OKRs to determine if experiments are successful or not and getting the whole team involved in quality and product ownership.
AI and the impact on QA
Given by Chhavi Raj Dosaj who proposed the question, can AI can the landscape of testing? Chhavi had a big topic to fill. I think his main point was a little lost because he spent too much time going over the basics and he wasn’t able provide compelling examples of where AI could benefit the QA process.
One of his examples was using an unsupervised learning algorithm to select the top manual test cases to execute in the next testing cycle based on past performance of said test cases. Anyone who primarily uses test cases to manage their testing work and reporting is using a pretty dated testing practice that doesn’t scale or make sense in the Agile world. You can read more about my thoughts on the end of manual testing. I also wish he asked me for feedback on his slide design before he got up on stage. There were some presentation sins committed in his slide deck.
dev Ops Testing Strategy
DevOps Testing strategy by Amit Kulkarni fell victim to the old bait and switch presentation sin. The title didn’t match up to the content. My key take away from this talk was, “there are tons of tools out there you can use in a continuous testing way”. There wasn’t any DevOps in this talk at all.
Performance Testing in CI
Andrey Pokhilko gave an engaging talk on Performance Testing in CI using opensource tools. I learnt the name of a new tool; Taurus. Which can be used on top of JMetre and is a little easier to use than the good old JMetre.
Bruce McLeod gave an engaging talk in business strategy and how it will transform testing. This was one of my highlights of the conference. Bruce goes over how testing needs to change from a culture of “protection” to a culture of experimentation using the Netflix chaos monkey as an example.
A few years ago I was lucky enough to win a YOW! diversity scholarship for a day of speaker training with Damian Conway. I’ve never had the opportunity to learn so much about public speaking and giving technical presentations in as condensed timeframe before. These are my hints and tips picked up from that training and other public speaking activities.
These following talks are good inspirations for better tech presentations
Damian Conway’s YOW! Night talk on Instantly better tech presentations
Vicky Brasseur’s 10 step program for great tech talks
Brainstorming your talk
Damian goes through a brainstorming technique to help form your talk. DO NOT start with slides, this is a sure way to waste time with creating your story as you iterate through it. First off, start with a pen and paper (or equivalent digital note taking technology), brainstorm all of the possible things you know about a topic. Pick 5 sub-topics you know the most about. Ask yourself, do these create a narrative flow? Do they form a story? If not, chuck one idea and pull in another that helps with the story. Then for each sub-topic brainstorm all of the things related to it, pick 5 from that list that forms a mini-story. You now have a good, narrative flow through your presentation and your slides almost write themselves, you could now put one word/phrase on each slide that reminds you of the story you are telling at each point and you’d have a pretty good presentation.
Think of your slides as a user interface, too much information will overload your audience ensuring they switch off and start browsing the web. Your audience doesn’t have the insights you have, that’s why they are in your audience. They want to learn, what do you want them to learn? Most often they won’t care about you as a person, they care about your content. Structuring your content around stories will help make your information easier to digest. Big walls of texts are a huge no no. Slides are free, why does all of your content have to be squished on 1 slide? I avoid dot points like the plaque, I can’t see any value they add being on slides.
Your text is too small
Always assume your text is too small, just because you can easily read it on your high resolution retina display doesn’t mean the person in the back of your audience is able to read it through a shitty 800×600 projector with terrible contrast. Also never put anything important at the bottom of your slide, how often have you strained to look over someones shoulder to try and read what’s on a slide?
Ditch that Intro slide
We (the audience) do not need to see what you look like, you are standing in front of us. We also do not need to know why you feel the need to justify why you are on stage. You already sold us with your title and description in the conference program. So please ditch that intro slide, it may switch off your audience before you’ve even really started. The audience is there to learn from you, just jump into the good stuff.
Build up to showing complicated diagrams/graphs rather than showing it all at once, you want to walk people through you idea not hit them over the head with it.
1 thought per slide
This is called Takahashi method, it’s awfully minimalistic but it forces you to use your slides as a crutch for your story telling rather than you relying on your slides too much. It’s worth experimenting with as an idea.
Screenshots of code work better than switching into an IDE/text editor, your code will always be too small with a text editor. If it’s a screenshot you also get syntax highlighting. Screenshots also help you to simplify the ideas you are trying to communicate.
Can’t avoid the complicated diagram? Fine, use opaque boxes and slide transitions to help guide your audience through the diagram. Take them on a journey. Here is an example using an XKCD diagram.
Demo’s are optional
I actually enjoy giving a demo first and then explaining how I got there, that way if it fails I still have content to continue with and I don’t end on a sour note. This way I always ensure I have enough time for the demo; which is often the fun part of technical presentations.
Nothing beats practice, if you can’t practice with colleagues, friends or meetup groups set up a TV with a picture of an audience and present to that. Or present to a mirror, or to your cat.
I enjoy performing. Don’t ask me why. I can’t explain it. You could say it’s something to do with the rush, or the perception of adding value or entertainment for other people. I want to tell you a few stories about my adventures in performing. Do you want to improve your performances? I’m available for free consultations on improving technical presentations.
During High School
I was involved with nearly every extra curricular activity I could sign up for. I was in the school concert band; I played trombone. I can legitimately say, “this one time in band camp …”. I was the fat kid in school, there weren’t many other kids fatter than I. I once got up in front of my whole school dressed up in a Santa suit and played Jingle Bells on the trombone. Talk about a nerve racking, getting out of my comfort zone experience. I got a laugh at least. I was in an Auslan signing choir (Australian Sign Language) and a singing choir too. In the signing choir we would often perform to retirement homes in the area and our signature song was, “I believe I can fly” by R Kelly. I could still sign to that song. What does a signing choir performance look like? Check out this example on YouTube;
Watching that makes me want to sign up to an auslan class and pursue deaf poetry.
I was also in a musical. It was called Wolfstock, it was a 1950’s themed musical about a 16 year old boy called Jay, his parents had sold his soul to the devil and had to get to wolfstock (aka woodstock) before the next full moon or else he would remain a warewolf. I played Wolfman Jack in act 2; a character based on the DJ host by the same name, I even had my own song. I’m sure the musical was terrible. My mum has it on tape somewhere. I’m sorry mum for putting you through all of my horrible performances in school.
I ran my own radio show on a community radio station called, “chat with an engineer”. I would interview engineers in our community and chat about the work they did. It was to help raise the profile of Engineering. I didn’t have the budget for the training course so I asked Engineers Australia if they’d paid for me to do the course. They did and I’m forever grateful for that. My biggest success was interviewing 2012’s Young Australian of the Year; Marita Cheng. She was visiting a high school as part of a Robogals visit and we were able to organise an interview.
I also started the Robogals Chapter in Tasmania. Robogals is a student run group who promote engineering and technology to young kids through lego robotics workshops with the goal of increasing female engineers. I taught robotics to over 1000 kids in tasmania in the 1.5 years I was involved with Robogals with next to no funding and while going through my first bout of chronic depression. I can’t understand how I was functioning, I wasn’t passing uni so let’s just say I wasn’t functioning very well. Teaching is another type of performance that I enjoy.
During my professional career the main performances I’ve been involved with are presentations. My most nerve wracking experience was getting up in front of the whole company during an all hands and talking about my struggles with depression. Getting that venerable in front of such a large crowd is another one of those big, “getting out of my comfort zone” experiences. It’s definitely made giving technical presentations easier. Interviews are another performance. A lot of people hate interviews, in a weird way I enjoy them. Having that opportunity to talk about my passions in software testing is what I enjoy. I am narcissistic. I remember doing a first year psychology 101 personality test during uni, I scored very highly on the narcissistic scale and I’m ok with that. It’s only an issue when it’s combined with a lack of empathy.
I’ve been involved with a few community bands since moving to Sydney. The Sydney homotones and Sunday Assembly being the main ones. I’m not actively involved with any now but I would love to join a community swing band. Or do some taiko drum classes. Or learn how to play the double bass. Garhhh, I can’t decide.
My favourite presentation has been my talk at YOW! Connected last year on using robots for mobile testing;
I was able to combine my passion for music, robots and mobile testing. #Winning at life.
I’ve collected a bunch of hints and tips on giving presentations. Reach out to me at sam[AT]thebughunter.com.au if you’d like a free consultation.