Categories
Software Testing

Why bother testing?

As a tester, I don’t practice selling my craft very often. This blog post is an attempt to put together my thoughts when asked the question, “why bother testing?”

Let’s talk about bugs

When was the last time you used some buggy software? How did it make you feel? Did it cause you to swear at your computer in frustration? We are surrounded by software full of bugs, some bugs are minor but some cause us to pull out our hair in frustration. Some bugs when discovered cause nightmare headlines to spread like wild fire. Buggy low quality software is hard to sell, maintain and keep people using. Your customers won’t rave about your products to their friends if they think it’s shit.

Your customers won't rave about your products to their friends if they think it's shit

Testing doesn’t improve quality

Testing itself doesn’t improve the quality of the product, but it can help highlight issues that could impact the perception of quality. Bugs will always exist in software, it’s impossible to test for every possible scenario every time before releasing to production (especially in a world of continuous integration/deployment) but if you knew about some the bugs in you products before your customers find them then you’d be in a better place to make informed decisions about what to release and when.

Testing is active discovery

Everyone does testing on some level, most of the time we are unaware of what we are doing until we discover a frustrating situation. Testing is a skill where you practice looking for quirks in software. Testers are constantly experimenting and observing the product and are well practiced in talking through their thought processes.

There are generally 2 activities people say they are doing when they talk about testing;

1 – Verify the product works as intended

2 – Actively go hunting for bugs

Activity 1 is usually where people talk about automation, when your product can be codified as “working as intended” you might be in a place to build some automation checking to help facilitate faster feedback. However you could build all of the automation checking in the world into your products but people can still think it’s shit. You users don’t care that your unit tests are less than 0.01% flakey.

Activity 2 can be easily practiced, it’s not like anyone knows where the bugs are in software until they are discovered. You can build things into your products that make it easier for people to report the bugs they find or you could do a chaos monkey approach to find out where your product crashes unexpectedly.

There are many other activities involved with testing but I want to keep this high level.

People say testing is risk mitigation

This is not true, testers may use their own internal risk radars to help guide their testing efforts but testing itself doesn’t mitigate any risks. I like the analogy; testing is like an x-ray, an x-ray on it’s own can’t tell you how healthy you are or prevent issues from happening but it’s a tool that can give you a snapshot in time so your doctor can make informed decisions about some element of your health.

Testing can help people answer questions like, “are we comfortable shipping this code to customers?”. Bugs are a fact of life but we can’t fix what we don’t know. Testing helps us discover these potential issues.

Categories
Uncategorized

dogfooding

{place holder text here}

please excuse the blank space I’m preparing these posts to support my agile australia talk as a way to generate online conversations http://agileaustralia.com.au/2018/

Categories
Uncategorized

Knowledge Sharing

please excuse the blank space I’m preparing these posts to support my agile australia talk as a way to generate online conversations http://agileaustralia.com.au/2018/

Categories
Uncategorized

bug bounties

{insert text here}

please excuse the blank space I’m preparing these posts to support my agile australia talk as a way to generate online conversations http://agileaustralia.com.au/2018/

Categories
Agile Software Testing Technology

Running a Bug Bash

For any solo tester out there I recommend leading a regular bug bash/mob testing activity. It’s an activity you can run at the end of a sprint/feature dev cycle. You invite the team, get some snacks/beverages together and get everyone testing for around an hour.

Setup

You might want to make sure you have good device/browser coverage by ensuring everyone is set up with data/devices before hand. You might also want to prepare some help docs to help people get started. I like to have a mindmap on coverage ideas prepared before hand so people have a visual indicator of what they are testing. I might also create a bunch of test accounts before hand and distribute them to the team.

I use this whiteboard for running bugbashes at Insight Timer; regression on the left, what’s changed recently in the middle and “other” things to consider on the right

Light Bug reporting

You should encourage light weight bug reporting;

  • maybe people write bugs on sticky notes
  • add bugs to a shared spreadsheet
  • or raise Jira bugs directly from Slack using a /jirio create bug shortcut

The focus should be on finding bugs, not getting caught up in how to report them. You can always clarify issues further after the bug bash if you need more information.

What to test in a bug bash

I have a 3 step heuristic for deciding what to test (think; guide or rule of thumb):

  1. what’s changed recently? Change introduces new risk. what features have been built? What code has been refactored? What libraries have been updated? Focus the team to test these areas first
  2. Regression testing. Have a checklist of at most a dozen scenarios of core functionality (maybe around registration, payments and the main stuff people do with your product). Ask people to consider these cases while doing 1), maybe ask people to put there name next to a case when they test it so you can get an idea of coverage
  3. What other testing should/could we do? Sometimes it might make sense to do a pen test bug bash session or a performance testing bash or a cross browser test bash but it doesn’t have to happen every time

Test in Production

If you can; test in production. There’s no environment quite like it.

Put on your ruby red slippers and repeat after me, "there's no place like prod"
Toto, I’ve a feeling we’re not in Kansas anymore – Wizard of Oz

But if you can’t test in production, make sure your test environment is stable, your data is set up and you’ve given your team the “Do Not Touch except for bug bash testing” request.

Post Bug Bash

Make sure to thank people for their time, count up the bugs and give kudos to your best bug hunter. Testing/Quality is a team responsibility after all. Send out a thank you email with results and award trophies. I’ve handed out this trophy of a rhino bug in resin on a trophy base to our number one bug hunter before:

rhino bug in resin on a trophy base
Bug Bash Trophy
Categories
Marketing Technology

Good morning World, what offends you today?

Ah the internet… is it just me or a things getting angrier lately?

Sometimes I can see how the internet makes people angry. E.G. there is a bit of a Uncle Bob vs Sarah Mei thing happening on twitter over how the word craftsman isn’t inclusive. I think it’s a reasonable point to raise, I’ve intentionally tried to stop using the word “guys” in my vocabulary here is an example tweet that demonstrates why I might try to avoid using those words;

A fond memory from my university days: I was seeking to understand why we can't use "mankind" or "men" for all cases. A wise woman asked me to name 10 famous men. I named 10 famous males, and no females. Her point exactly. That's the problem with using craftsmen for all cases.
https://twitter.com/srogalsky/status/991431460681854976

So I’ve come to realize words like guys and men aren’t really all that gender neutral/inclusive after all. I personally didn’t think about the word craftsman being noninclusive before this point. I do talk about the craft of testing and will continue to talk about it. But I’ll make sure to try to catch myself if I’m about to use the word craftsman. Huzzah, the power of language; it changes and adopts to new cultural expectations. Original tweet for reference;

Uncle bob says; "Rude. Craftsman is a gender neutral term"
https://twitter.com/unclebobmartin/status/990411971106426880?s=19

My takeaway here; it’s hard to be polite on twitter. Both sides are being rude (but their online personas are built around this, I imagine these people are too busy to give a shit over how the world interprets their tweets and rudeness/bluntness can help drive conversation, it motivated this blog after all and many people are now re-assessing the word craftsman). But I’d have to side with Sarah on this one and I think Uncle Bob was just a little ruder than Sarah here. Do you disagree with me on this one?

Other things I see are just riding the “anger” train to get people riled up. People on both sides of the equation are guilty of over reacting though. Then there’s this teenager who found a Chinese dress in a thrift store and wore it to prom;

How did this become news? I was given a Japanese Yukata from a mayor of a Japanese town, do I have to be concerned when I wear that now? I think that aj+ piece is just asking for that knee jerk emotional reaction, I’m actually a little disappointed in aj+ about this. It reminds me of an article I saw come up in my facebook feed from a pop culture media outlet about wonder woman’s hairless armpits; about how angry feminists were critiquing the trailer and asking, how does wonder woman shave? This was clearly click baity because 1 “okay the Wonder Woman movie looks rad but why does Diana have clean shaven armpits” is hardly angry (or did I miss something?) and the article was clearly shared on facebook for the knee jerk facebook reaction, “dam special snowflake feminazi’s ruining a good movie like wonder woman”. I stopped following the pop culture media outlet because of it. Does anyone remember how angry Australia got over Yassmin’s, “Lest. We. Forget. (Manus, Nauru, Syria, Palestine…)” tweet? I definitely wasn’t proud to call myself Australian after the hate she received after that.

However anger is pretty effective at engaging people, it’s a common marketing tactic now (get people angry and talking about it). BUT I think it causes people to get into a “defensive” thinking mode and it can be really hard to reason through that knee jerk emotional reaction.

Now I’m not a christian but when I feel that anger bubble up and inspire me to get all keyboard warrior I take a moment to reflect and think, “how would Jesus react with love and compassion in this situation?”. Jesus is my role model in these kind of situations. How do you catch yourself before you respond in anger? Or maybe I should harness my anger to drive discussion? Like James Bach does in this blog about Machine Learning tools in software testing are bullshit. It definitely has a lot of people talking about it.

Is it just me or are there more angry people out there?