Categories
Mobile Testing Software Testing

Bugasura and exploratory Mobile Testing

A few weeks back, Pradeep Soundararajan; founder of Moolya testing and I were having a conversation on twitter on test strategies for mobile apps. He suggested trying Bugasura for running a bag bash.

Bugasura is an android app and a chrome extension. it helps with keeping track of exploratory testing sessions and comes with screenshot annotation and jira integration. 

Here are a couple of screenshots of the android app in action, being used for an exploratory session on our test app.

Bugasura Flow

First I selected the testing session:

While I’m testing I see this Bugasura overlay which I can tap to take a screenshot and write up a bug report on the spot:

Here’s their reporting a bug flow:

And here’s a testing report after I finished my exploratory testing where I can push straight to Jira if I want:

Here’s the sample report link (caveat, the screenshots attached to the bug are now public information on the internet, so there’s a privacy concern right there), but OMG, the exploratory session recorded the whole flow too (so a developer could see exactly what I did to find that bug).

Flow Capture

Here’s that bug report in chrome paused at screen 13 out of 18:

On a side note, I love having these Jerry Weinberg (who wrote perfect software and other illusions about testing) and Elisabeth Hendrickson (who wrote Explore It!: Reduce Risk and Increase Confidence with Exploratory Testing) quotes sprinkled through out the app.

Some caveats I’ve found so far; the test report is public (not private by default), you wouldn’t want to include screenshots of private or confidential information.

Bugasura only works on android/chrome. There isn’t an ios version but I guess with some remote device access running through Chrome it could work? We use Gigafox’s Mobile Device Cloud at work to access a central server of mobile devices and I imagine Bugasura could work with it.

Also I think they may have misspelt Elisabeth’s name in her quote.

This blog post reflects my opinions only and do not reflect the views held by my employer.

Categories
Mobile Testing

Beta releases for mobile apps

My team is going through a beta release for our mobile app to get early feedback. We’ve noticed that our android app is struggling compared to iOS. It seems that having an extra hurdle with signing up for the android beta program impacts installations. Naturally we’d expect the android engagement to lag a little behind iOS based on the Aussie mobile usage market analysis but there is still a significant drop.

We have 428 iOS installs, and 99 android installs. That’s a 19% Android installation rate. We have roughly 75-80% successful registrations and return log ins once people actually figure out how to install the app.

Google Groups vs Test Flight

We are using google groups to manage the distribution of the android beta app and because it’s harder to use than test flight for iOS we’ve gotten less installations. It’s fascinating how an extra hurdle in the sign up process can impact installations.

Return Logins

Our android numbers appear to be higher than usual here but I think it’s to do with the timeframe I’m collecting these numbers over. We’ve had a few people install the android app before we officially started the beta release and I think they’ve been counted in this statistics.

Tools

We are using adobe analytics, test flight and using a closed test track via google groups to help us manage the beta app releases and to gauge engagement metrics. You can read more about our general mobile app test strategy here if you like.

What were your experiences going through a beta release process? What worked well? How did you improve the process?

Categories
Software Testing

The Y2K bug in the year 2038

Do you remember how the world freaked out over the potential Y2K bug when the year was changing from 1999 to 2000? A large mitigation factor was a huge outsourcing effort to India and it helped to establish India as the global IT giant it is today. So when not many bugs eventuated it was a bit anti climatic. 


Globally; $308 billion dollars was spent on compliance and testing and it helped build more robust systems that survived the system crashes from the 9/11 terrorist attacks. 


Well the y2k bug would only impact systems that used 2 digits to represent the year, i.e. using the DDMMYY format to save on memory.

And the world updated this and their systems every where. Businesses did a pretty good job of patching that bug before it became an issue. Bugs still did come up but the world didn’t end.

Sometimes the fix was to push it out

Some fixes people implemented was to make 00 to 20 stand for 2000 and 2020 respectively. That’s only pushed out the problem and we’ve had some cases this year of this bug coming into effect. You can read more about this 2020 Y2K bug here.

There’s another bug for 2038

However did you know there is another Y2K bug scheduled for the year 2038? Basically the way our current 32 bit computer systems count time is the number of seconds since 1970. In the year 2038 we get a bit overflow issue and that counter resets to zero. This is more likely to impact cheap embedded systems with limited memory or old legacy systems.

If we switch to a 64 bit counter, our sun will explode before we get the same issue. It’s like going from ipV4 (we are already running out of ip addresses) to ipV6 for the internet.

IPv6 – Let the Sky Fall!

2038 might feel like a long way away but here’s a story about pension payments breaking in 2018 due to this bug (20 years before this bug is meant to occur):

Is your system at Risk?

So, are any of your legacy systems at risk? Do you think you’d have them migrated to cloud infrastructure before this issue occurs?

How about setting up a test environment that’s configured to be 6 months ahead of production? Would that give enough time to find and fix these issues when they occur?

Categories
Critical Thinking Mobile Testing Software Testing

Mindmaps and Heuristics for testing

I like to use mind maps to help me test. Mind maps are a visual way of brainstorming different ideas. On a previous mobile team I printed and laminated this mind map to bring along to every planning session to help remind me to ask questions like, “What about accessibility? Automation? Security? or Performance?”:

As I go through exploratory testing (or pair testing), I’ll tick things off as I go and take session notes. Often this will involve having conversations with people, sometimes bugs are raised. 
Here is a quick mind map I’ll use for log in testing:

Heuristics for testing

This mind map approach can be combined with a heuristic test strategy approach or a nemonic test approach. Heuristics is a rule of thumb that helps you solve problems, they often have gaps because no mental model is perfect.

SFDPOT is a common nemonic that was developed by James Bach; who also developed Rapid Software Testing; a context driven methodology. James Developed his RST course with Michael Bolton

In SFDPOT each letter stands for something that is meant to help generate testing ideas, they are; Structure, Functions, Data, Platform, Operations and Time. Here’s someone’s blog on applying SFDPOT to a mobile app testing approach.
I tend to use all of these different approaches as part of my exploratory testing practices.

More resources

If you are interested in reading about exploratory testing Elisabeth Hendrickson has written this book called Explore It!, reduce risk and increase confidence with exploratory testing. Elisabeth also has this nightmare headlines game which is a fun tool for brainstorming potential error cases.

Your team could also go through the process of creating your own nemonic/mindmap to help you have consistency across different testing styles, maybe something that made sense for your context? 

You could also use mindmaps to feed into the different feedback loops you’d like to have in your team. You can read more in this mobile test strategy blog.

Categories
Conferences

YOW! Sydney 2019

YOW! Sydney 2019 has come and gone and what a conference it was. Here is my summary of the talks I attended (with sketchnotes). You can access a PDF of my sketchnotes here. Or find them all on twitter under #YOW19 and #Sketchnotes. Hopefully I’ll see you at the 2020 conference.

If you are interested in starting with sketchnotes, I have hints and tips here.

Keynotes

Day 1: Gene Kim and The Unicorn Project

Gene Kim is a well established author and consultant in the DevOps space. He’s written The Phoenix Project, The DevOps Handbook and now his third book The Unicorn Project. I’m looking forward to reading the next one in the series. You can see a similar talk from DevOps Days Portland on youtube here:

Day 2: Troy Hunt and we are all pwned

Troy Hunt always gives an entertaining presentation and he’s a security expert from the Gold Coast, Queensland. He runs a website called, Have I been PWNED and the answer is most definitely always yes. He’s got a similar talk from NDC Sydney here:

Other Talks

Graphs & Investigative Journalism by Michael Hunger

Quantum Computing by Matthew Keesan

Growing your own Design Heuristics by Rebecca Wirfs-Brock

Interaction Protocols by Martin Thompson

Level Up Quality, Security and Safety by Todd Montgomery

Frictionless Frontends by Mandy Michael

Full Stack Accessibility by Larene Le Gassick

Standards by Nicola Nye

Inventing the Future by Allen Wirfs-Brock

Technical leadership matters by Dave Thomas

How I CTO by Gil Tene

Categories
Technology

Women in Tech

When I start talking about diversity in tech there’s a few points I usually like to make. Because I come from both a tech and engineering background there is a lot of overlap with women in Engineering roles and women working in tech. This blog post will compare diversity in tech and engineering.

We nearly had 40% female graduates in Computer Science

Traditional engineering fields such as mechanical engineering have always had low female participation rates at the university level. Where as computer science didn’t have that problem. A common reason why people think women stopped studying computer science was the personal computer came out in the 80’s and was marketed exclusively to boys. This was also around the time when Lego changed it’s marketing towards exclusive “boys toys”. (Lego was struggling financially before this point). This video below compares Lego’s 80’s advertising to current day advertising.

Some engineering fields have more diversity

What draws women to study Environmental and Biomedical engineering over Mechanical Engineering? I’d say it’s a perception thing, as a smart young women entering university, the amount of courses you could study is almost endless. You want to do something that feels like it’s going to benefit society as well as stimulate you intellectually.

Environmental Engineering definitely sounds like it’s going to benefit society more than Petroleum Engineering… Just saying.

Women and international students pass engineering degrees at higher rates

Once women decide to study Engineering they are more likely to see it through to successfully graduate. If women had some sort of innate ability that made them not good at engineering, you’d expect this number to be lower.

Some Muslim countries have better diversity in Engineering

Malaysia boasts about 30 – 40% of women engineering graduates

https://www.asee.org/public/conferences/106/papers/22417/view

People tend to associate Muslim countries with oppression of women, however choosing a STEM career is seen as a way for setting yourself up for financial independence in a lot of these countries. There’s a different cultural perception of value for these careers. People in these countries don’t think that maths is a skill you a born with and only boys are good at it. They believe to be good at this skill you need to study hard. There is a blog post on cultural influences here.

Women leave the field more frequently than men

To become an Engineer you have to be a smart cookie, it’s not an easy thing to do. I know this because I tried and failed in my efforts. But once you start working there’s lots of issues that constantly chip away at your sense of satisfaction in work. There’s also tons of non engineering fields out there that value transferable engineering skills, can be just as rewarding on an intellectual level and offer better work life balance.

Research also shows that women are disproportionately likely to move away from the most technical career paths and toward roles that involve technical supervision or management as their careers progress.

https://hbr.org/2018/11/the-subtle-stressors-making-women-want-to-leave-engineering

We have a women to leadership problem

If we look at a more traditional female job (teaching), 74% of the workforce are women but 49% of the principal’s are male. We naturally promote more men to leadership positions.

https://docs.education.gov.au/system/files/doc/other/ntwd_data_analysis_report.pdf

Diverse Companies perform better financially

I’ll leave you with this TED talk on how diversity is good your companies bottom dollar.

Categories
Conferences Mobile Testing Presenting Technology

DevWorld 2019

I recently attended /Dev/World in Melbourne this week. This is Australia’s only iOS developer conference. Here is my summary of the conference.

Photos from the conference on Twitter under the hashtag #DevWorld

Themes

There were a few themes that gleaned from the talks. These were;

  • Swift is the most talked about language in this space
  • People are still using Objective-C
  • Cross platform remains a hot topic (from iOS to mac, React Native and Flutter)
  • Augmented Reality is still a cool toy
  • How to build engaging experiences in your app
  • Game design stories
  • Testing

Sketchnotes

Here are my highlights:

You can access all of my sketchnotes on google photos here.

My Talks

I presented on swift UI & wearables using my sample poo tracking app as the basis for it. You can watch a similar talk from my twitch channel. You can also access my slide decks here. I also gave a lightning talk on how to do sketch noting. You can read this blog post if you are interested in learning about sketch noting. I also have this blog post on Right To Left bugs which was inspired from my talk if you are interested in reading more.

Recordings

Eventually the talks will be up on youtube under the AUC_ANZ youtube channel. You can access previous years talks there too.

Categories
Design Mobile Testing Software Testing Technology

Right To Left design considerations for mobile apps

We truly live in a global and inter connected society. But have you tested your app using a Right to Left (RTL) language such as Arabic? This blog post is a reflection on some of the design considerations to keep in mind when accomodating this.

Why does this matter?

Arabic is one of the top 5 spoken languages in the world with around 3 hundred million speakers and it is the third most spoken language in Australia. Even if you only release apps for the Australian market someone out there will have Arabic set as their default device language. It’s ok if you haven’t translated your app, but you should check that these people can still use it.

How do I test this?

Android

Enable developer options and select “Force RTL layout direction”. On My Samsung S10 this is what my screen and options look like after enabling this option:

iOS

In Xcode you can change the build target language to a Pseudo RTL language to see how your app renders in this way without having to change the language on your device.

Number pads

You don’t actually need to render your key pads in Right To Left, in fact it’s actually more jarring to render numbers in a RTL arrangement because ATM’s and phone pads are left to right in Arabic. Most Arab’s are use to globalised number pads. Samsung has an in-depth article on when RTL should be applied.

When I have RTL rendering set on my android phone, the log in pin screen and phone call functionality is in LTR. However some of my banking apps render their pin pads in RTL.

Common RTL Issues

I was pleasantry surprised to find out how many of my apps weren’t broken when I switched to RTL rendering. Facebook, twitter and email still look very good. Some apps (like my calculator) do not make sense to render RTL and they remain LTR:

Bug One: Overlapping labels

You will have to watch out for when labels overlap like in the domain app here:

Bug Two: Visuals doesn’t match written language

And when your text is rendered RTL but the visual cue is still LTR like in the shade bar for representing countries visitors to my blog in this wordpress statistics view:

Bug Three: Menu’s that animate from the side

In the app I’m helping build, the side menu renders pretty funkily in RTL mode, I can’t show you a screenshot of this behaviour but it’s probably the quirkiest RTL bug I’ve seen. If you find an app with bad side menu behavior in RTL please share your screenshots with me.

But here are some screenshots of the CommSec app on android (LTR on the left and RTL on the right for comparison)

Bug Four: Icon’s aren’t flipped

Often icon’s have a direction associated with them like the walking person when you get google maps directions. Sometimes it can look a little odd when they aren’t flipped correctly (as if they are walking backwards).

Have you seen these bugs before?

Please let me know your thoughts or experiences in supporting RTL languages. I’d love to hear your stories.

Categories
Google Cloud Technology

Serverless Days Sydney 2019

I attended Serverless Days in Sydney Today. Overall it had a good sense of community, the venue and food was top notch. This is a community run conference and there’s always good representation from the main serverless cloud providers here. *Cough* Google/AWS/Microsoft *Cough*. Alibaba made an appearance too.

Are we at a serverless tipping point?

Jeff Hollan – a Senior Program Manager for Microsoft Azure functions doesn’t think we are there yet but we are getting close and you can help by sharing your learning and feedback. Here is my getting started with google cloud serverless functions on google cloud blog post. You can check out one of Jeff’s previous talks on Azure Functions for enterprise here:

Can we apply a serverless mindset?

Ben Kehoe – a Cloud Robotics Research Scientist at @iRobot gave us lessons learned in applying this mindset to more than just web applications. He goes over how as engineers we need to shift our mindset to focus on delivering business value and not get caught up on feeling like the centre of the business. Any code your write is a liability. That’s why I advocate for lean test code that adds value.

On a side note, anyone who gets to work in robotics professionally is super cool in my books. My favourite presentation is one where I present Tappy McTapface – a robot for mobile app testing. You can watch a previous talk by Ben on a similar idea here:

Did you miss out on the action?

Some of the speakers have given their presentations at previous conferences/meetup groups. Like Jessica Flanagan at YOW! Data this year. And Denis Bauer at YOW! Perth last year.

I Previously live streamed the Serverless Sydney meetup group and the Node user group on twitch (however these video’s are only stored on twitch for up to 60 days post live stream).

Serverless Icon Showdown

Who has the best logo? There’s Google Cloud Functions, AWS Lambda Functions, Microsoft Azure Functions and Alibaba Cloud Function Compute. Handsdown I think Microsoft Azure functions is the best logo here.

Are you using Serverless in production yet? what are some of the challenges you are facing? How should testers get up to speed with this technology?

Categories
Google Cloud Technology

Getting started with Google Cloud server less functions

Do you know how many lines of code you need to deploy a google cloud server less function?

Short Answer – 4

exports.helloWorld = (req, res) => {
  let message = req.query.message || req.body.message || 'Hello World!';
  res.status(200).send(message);
};

https://cloud.google.com/functions/docs/quickstart-nodejs

Wait, what’s a server less function?

Have you heard of micro services? This is like that but even smaller. What’s the smallest service you could possible deploy? a single function. Server less still uses a server, but the infrastructure is abstracted away by your cloud provider of choice. If no one is hitting your API, there’s no server running and costing you money. Cloud providers spin up an instance when needed as soon as something calls your function.

If you are interested in learning more, you should go to Severless Sydney on Saturday the 27th of August. Tickets are $69. https://sydney.serverlessdays.io

How do you integrate a server less function with a mySQL backend?

const mysql = require('mysql');
const connectionName =
process.env.INSTANCE_CONNECTION_NAME || 'boglogger:australia-southeast1:boglogger2';
const dbUser = process.env.SQL_USER || 'DEMOUSER';
const dbPassword = process.env.SQL_PASSWORD || 'DEMOPASSWORD';
const dbName = process.env.SQL_NAME || 'bog_logger';
const mysqlConfig = {
connectionLimit: 1,
user: dbUser,
password: dbPassword,
database: dbName,
};
if (process.env.NODE_ENV === 'production') {
mysqlConfig.socketPath = `/cloudsql/${connectionName}`;
}
// Connection pools reuse connections between invocations,
// and handle dropped or expired connections automatically.
let mysqlPool;
exports.logBog = (req, res) => {
// Initialize the pool lazily, in case SQL access isn't needed for this
// GCF instance. Doing so minimizes the number of active SQL connections,
// which helps keep your GCF instances under SQL connection limits.
if (!mysqlPool) {
mysqlPool = mysql.createPool(mysqlConfig);
}
mysqlPool.query('SELECT NOW() AS now', (err, results) => {
if (err) {
console.error(err);
res.status(500).send(err);
} else {
res.send(JSON.stringify(results));
}
});
mysqlPool.query('INSERT INTO bogs_logged(time, device_uuid, type, amount, floater) VALUES(NOW(), "e5c15033-42b3-487d-b6c8-17a1b592e9bd", 2, "S", TRUE)', (err, results) => {
if (err) {
console.error(err);
res.status(500).send(err);
} else {
res.send(JSON.stringify(results));
}
});
};

https://cloud.google.com/functions/docs/sql

How about getting JSON out of a request?

const escapeHtml = require('escape-html');
exports.helloContent = (req, res) => {
  let name;

  switch (req.get('content-type')) {
    // '{"name":"John"}'
    case 'application/json':
      name = req.body.name;
      break;
  }

  res.status(200).send(`Hello ${escapeHtml(name || 'World')}!`);
};

https://cloud.google.com/functions/docs/writing/http

Combining HTTP and MySQL

Now let’s combine the two, let’s build an API that logs your device ID and your poo type into a google cloud instance:

const escape = require('escape-html');
const mysql = require('mysql');
const connectionName =
process.env.INSTANCE_CONNECTION_NAME || 'boglogger:asia-northeast1:boglogger-prod';
const dbUser = process.env.SQL_USER || 'addBogs';
const dbPassword = process.env.SQL_PASSWORD || '';
const dbName = process.env.SQL_NAME || 'bog_logger';
const mysqlConfig = {
connectionLimit: 1,
user: dbUser,
password: dbPassword,
database: dbName,
};
if (process.env.NODE_ENV === 'production') {
mysqlConfig.socketPath = /cloudsql/${connectionName};
}
let mysqlPool;
exports.addBog = (req, res) => {
let device_id;
let type;
if (!mysqlPool) {
mysqlPool = mysql.createPool(mysqlConfig);
}
switch (req.get('content-type')) {
// '{"device_id":"123", "type": 2}'
case 'application/json':
device_id = req.body.device_id;
type = req.body.type;
break;
}
res.status(200).send(BogLogged ${escape(device_id || '123')}!);
mysqlPool.query(INSERT INTO bogs_logged(time, device_uuid, type) VALUES(NOW(), "${escape(device_id)}", ${escape(type)}), (err, results) => {
if (err) {
console.error(err);
res.status(500).send(err);
} else {
res.send(JSON.stringify(results));
}
});
};

https://github.com/BugHunterSam/BogLogger/blob/master/day2Stream21-07-2019/severlessAPIAddBogs.js

My API is in production if you hit this API with a POST Request

https://asia-northeast1-boglogger.cloudfunctions.net/addBogs

With JSON

{

"device_id":"123abc",

"type": 3

}

I will log your entry in a mySQL backend

When you deploy that function from above you will need a few packages in your dependencies in your package.json file:

{
"name": "sample-http",
"version": "0.0.1",
"dependencies": {
"mysql": "2.17.1",
"escape-html": "1.0.3"
}
}

If you want to run your own API, you will need to create a google cloud SQL instance and add a user. You can use these commands to create a table and user.

--create tables and insert some test data
CREATE TABLE bogs_logged(time DATETIME NOT NULL, device_uuid VARCHAR(36) NOT NULL, type INT(1));

--create addBogs user
CREATE USER 'addBogs'@'%' IDENTIFIED BY '********';
GRANT SELECT, INSERT ON bog_logger.bogs_logged TO 'addBogs'@'%';

https://github.com/BugHunterSam/BogLogger/blob/master/day2Stream21-07-2019/SQL-users-and-triggers.txt

What’s the point of all of this?

I now have a server less API in production for my mobile wearable app to track your poo. Next Steps is building the UI!