A quick google of ‘What is software testing’ will give you 75,300,000 results but not much of an answer. On the surface level, we can surmise that it is about testing software but in reality; it’s more than that.
Software testing is an activity that everyone does on some level. E.g. when you interact with a product like Facebook you are helping to test it. You have certain expectations of how the scrolling behaves and what type of content comes up in your feed. You are probably a part of several on going experiments that Facebook conducts and you’re scrolling habits are also helping shape their performance testing approach.
Q. Hang on, if everyone does it, why do we need dedicated software testers?
A. Justifying the value I add to a project can be challenging sometimes but let me try to go into more detail. There are some very successful companies out there that don’t hire dedicated testers but their context and software development practices can help mitigate some of the risks associated with their products.
Software testing is about reducing the risks associated with software development. Software development is, at its core, a business venture. The software is created to solve an issue that the end user is willing to pay money for. Each element within the development lifecycle is working towards providing a marketable solution to that original problem. Software testing serves to highlight any risks that might impact on the value of the product.
The software development cycle is a continuous iteration of identifying the customer’s problems, developing the solution, ensuring the solution works, and selling the solution. Software testing ensures that our apps, websites and software does what it’s supposed to do whilst not doing what it shouldn’t. It is an integral part of any successful software project.
Table of Contents
Planning
Good testing can be completed with very little “formal” planning procedures if a skilled tester is involved. You can see an example test report generated from 1 day of testing from James Bach for testing a medical device.
Often the waterfall model of software development is associated with exhaustive requirement gathering and testing planning even before the software is designed. On the contrary, the agile manifesto focuses on providing working software over comprehensive documentation. The whole Agile vs Waterfall is a conversation for another blog.
When working within the agile methodology we can move away from explicit and highly specific planning and into the rapid software testing realm. James Bach and Michael Bolton are pioneers in the Rapid Software Testing space. This is a context-driven software testing methodology that works on the premise that the testing team will have an understanding of the business problem that the product is solving.
This Rapid Software Testing method allows the testing team to conduct a more informal, risk-based approach to planning the testing of software. Sam explains how to take a risked-based approach to planning UI tests in her /dev/world presentation.
Exploratory Testing
Exploratory testing* is where a tester interacts with software to compare it with their mental models of expected outcomes.
Exploratory testing is a part of all testing regimes because it focuses on human-based feedback. Having people run through the software with intention of finding brokenness helps to highlight any design, UI or experience elements that are confusing or hard to use. This is something that automated testing scripts aren’t able to handle.
Exploratory testers will explore how well the software solves the end-users problem. They will highlight any high-risk issues that will impact the value of the product. Skilled exploratory testers will work on a continuously iterative testing approach where the exploration can help form a plan that guides the direction of future testing efforts. This continuous feedback loop allows testers to improve the quality of their testing rather than just the quantity.
Exploratory testing is based on human observations. This is incredibly useful for user experience focussed testing but less practical for testing that needs to be repeated over time or scaled up to assess performance.
Whilst the benefits of having human based feedback far outweigh the cons it is important to note that human error will always occur. Hopefully using a risked-based approach with an experienced exploratory tester will negate some of these issues before your users find them.
Automated testing
Automated testing (sometimes called checking) involves using software tools or scripts to compare the expected results for how a program should run with actual outcomes. It leverages the computational power available today to automate testing that would have to have previously been conducted manually.
For example, an automated testing suite may be used to check that ‘a + b = c’. To test this we could run a sample of known numbers through the function and ensure that the outcomes were as expected.
This is an extremely simplified example of automated testing but this idea can be expanded to include all of the predefined actions within a software.
Automated testing has numerous benefits including increasing the amount of test coverage available within a budget or timeframe, automating basic or low-level tasks, reducing human error when checking outcomes, reusable tests and leveraging the computational power of machines.
If automated testing suites aren’t maintained in line with the development cycle it can lead to outdated tests that return incorrect outcomes. This can lead to less secure testing coverage and a loss of team morale and trust in the testing cycle. Additionally, automating poorly designed tests only increases the rate that these misaligned tests are run.
Automated testing has been hailed as the holy grail of the software development world of late. It fits in well with the continuous integration methodology and allows for quick iterations of products. However, it’s important to keep in mind that automated testing isn’t a magic bullet. Instead, automated testing should be used in conjunction with other tools to gain more time for your testers to focus on high-value tasks that are outside the scope of automation.
Unit Testing
Unit testing is about isolating the components in your software and testing them individually.
Unit testing is a powerful tool when used early in the development cycle. This is a tool often employed by developers to ensure that the code they’re writing works.
Like all of the tools that we have explored today unit testing in only one option that a development team will employ during as part of their testing process.
Conclusion
Anyone doing testing has a plethora of tools available to them when conducting a thorough testing cycle. Often we get caught up in the discussion of Exploratory Testing vs Automated Testing but we need to bring this back to what is the tool that will best fit this testing scenario. I’ve only scratched the surface here of what can be involved with software testing.
Software testing is a continuous feedback loop. Everyone does testing on some level, and one of the goals of testing is to highlight any issues or risks that might impact the perceived value of the product. Ultimately we want to help deliver quality products on time and on budget.
*The intricacies about whether exploratory testing should be referred to as ‘testing’ has been discussed extensively within the software testing industry. Please note: for ease of understanding Sam will be referring to all testing without scripting as ‘exploratory testing’ within this article.
**Sam commissioned Stefanie C. from upwork to help write this article, there might be a follow up blog on Sam’s experiences using freelancers to help outsource work later and why she would do this
Nice, well written post.
I would make a few changes:
I think of exploratory and automation as different dimensions. They aren’t really different tools for the same problem. They solve different problems. They aren’t interchangeable. (I think there are deeper reasons why exploratory is not essential.)
I don’t think people who choose automation make a conscious decision to use automation. I think most people don’t have an understanding of exploratory testing. As a result some choose automation others choose process, e.g., test plans, test cases.
I would also be careful of the idea that everyone tests. It’s one thing to find problems while you are using the software. It’s another to make inquiries to find unknown problems. I think most people *can* do the second. However it is not straight forward. It also requires a high level of creativity, among other skills.