Gene Kim is a well established author and consultant in the DevOps space. He’s written The Phoenix Project, The DevOps Handbook and now his third book The Unicorn Project. I’m looking forward to reading the next one in the series. You can see a similar talk from DevOps Days Portland on youtube here:
Day 2: Troy Hunt and we are all pwned
Troy Hunt always gives an entertaining presentation and he’s a security expert from the Gold Coast, Queensland. He runs a website called, Have I been PWNED and the answer is most definitely always yes. He’s got a similar talk from NDC Sydney here:
Other Talks
Graphs & Investigative Journalism by Michael Hunger
Quantum Computing by Matthew Keesan
Growing your own Design Heuristics by Rebecca Wirfs-Brock
Interaction Protocols by Martin Thompson
Level Up Quality, Security and Safety by Todd Montgomery
Today I gave a talk at TiCCA (Testing in Context Conference). The talk topic was tails of fail – how I failed a quality coach role. It’s a story of how I tried out this quality coaching thing but I didn’t pass probation. You can access the slides here. I used slido to manage the questions at the end of the sesion.
At the end of the day, quality coaching can be hard to demonstrate value add.
Will you answer all these questions offline ?
Yes, this blog post is an attempt to answer all of the unanswered questions that were raised. Thanks Richard. First of all, a bit of context that was missed in my intro. I’m currently a Test Analyst at a superannuation company, I don’t technically have coach in my title but I’m also growing my side business where I provide training and workshops for teams in testing skills. This might have caused some confusion with the questions.
What does a quality coach’s typical day look like?
When I was at Campaign Monitor, my day would start with a stand-up and seeing what items needed focus on for the day. The team might have a work item that needed a bit of testing and I’d be available to pair test with that developer if needed. Some days we would run workshops (training for quality champions; developers who wanted to improve their testing skills) or bug bashes (these were generally once a fortnight).
What are the differences between a quality coach and an agile coach and a test coach?
An agile coach is a facilitator, often scrum certified (but not always). They are usually more focused on helping the team collaborate more effectively over improving the teams quality/testing practices. I don’t see much difference between a test coach vs a quality coach. You can use the words that make sense in your context.
Are there any drawbacks to using a quality coach practice?
Yes, when you are encouraging people who prefer to build things workout their testers mindset you aren’t going to get as focused attention as someone who has spent their career practicing their testing craft.
Also, you might have some really technical testers who aren’t interested in coaching/leadership skills. You shouldn’t expect everyone to want to become a coach and that’s fine too.
What are the benefits to the organisation of the assistance/coach/advocate model?
If your company believes that quality is a team responsibility a coach can help lift the testing capabilities of a team. If you need a bit of focus on quality (maybe you have lots of customers complaining about bugs and it’s costing you big $) but you don’t know how to get your engineering teams to prioritise bug finding as well as build new features a coach could help here. There isn’t a great deal of training out there in how to be a good tester, it’s not as easy as sending your devs off to a 3 day course and bam they are master bug hunters.
If everyone is responsible for quality, is anyone really responsible for quality?
You could always say the CEO or CTO are fundamentally responsible for quality. Maybe have a Chief Quality Officer (CQO)? Maybe they’d just become a scapegoat for all of the problems you face in production? The testing teams themselves aren’t responsible for quality if they can’t easily build quality in either.
What is a good team to quality coach ratio?
Depends on the team/company. When I was at Campaign Monitor we had 2 testers to roughly 50-ish engineers. Hence why we were using the quality champion model to help get more quality reporting from teams. We physically couldn’t sit with all 6 teams at the same time to understand their pain points. I’d prefer a 1 coach to 1-2 cross functional teams. Being embedded and focused on one team of roughly 8 people would work for me.
What are the challenges you faced while quality coaching?
Clearly articulating how I add business value that aligns with my own intrinsic motivations and interests. I don’t think I’ve struggled with convincing developers they need to do more of their own testing.
Soap Opera Testing is a dramatised method used for testing your business processes. You might want to try it for a super-condensed and thorough way of highlighting bugs. And because it’s fun. Embrace the drama.
Origins
Cem Kaner has been writting about scenario testing for a long time. He published this article on ‘an intro to scenario testing’ and Hans Buwalda presented on ‘soap opera testing’ nearly 20 years ago 😱. They’re both serious tester dudes and this stuff is legit.
How Does it Work?
You might start with a brain storming session with your sales or customer support team. Ask them for stories about things your users have done. Not just the ordinary things, but also some off-the-wall and crazy things. What you’re looking for is drama.
It might help to sketch out the story briefly. Write down steps that are essential, or those that you might make a mistake on. Cem Kaner gives some practical tips here, although you definitely don’t need to read all 500 pages.
Kaner’s Introduction to Scenario Testing is a bit more bite sized and describes the five main points your scenario needs. Namely that it’s a story, it’s credible, it will test the program in a complex way, the results are easy to evaluate and stakeholders will see the point of fixing the bugs identified.
A scenario is a hypothetical story, used to help a person think through a complex problem or system.
You then run a test exercise using the characters and scenarios from a soap opera, and analyse the results. You can do this as many times as you want, with as many different scenarios.
Use whatever soap opera you like. We make no judgements. Although fair to say that if you use A Country Practice, you’re showing your age and nobody will know what you’re talking about.
Let’s Soap Up
Here’s an example of Soap Opera Testing using The Simpsons. The program being tested is a mortgage loan application.
Let’s say Homer Simpson wins the lottery, and decides to apply for a second mortgage, for an investment property. Just as the paperwork is about to go through, Grampa Simpson burns his apartment down.
Investment property
Homer decides to help him out with the cost of a rental, meaning he needs to change the deposit he’ll pay on his investment property. Homer signs the amended paperwork but he signs it incorrectly.
Then his application is declined because even winning the lottery doesn’t give you a good credit rating overnight. The Simpsons’ next ‘diddly-door’ neighbour Ned Flanders offers to help Homer out. He’ll put in the 10% deposit.
Ned lends a helping hand
His own house is 90% paid off so it’s no big deal to him, and it will help Homer get around his bad credit rating. The Simpsons’ house is 50% paid off, and they’re putting down a 90% deposit, using Homer’s lottery winnings, and leaving some bowling money left over.
Side investment
They’re about to go to the bank and lodge the paperwork, when Homer’s half-brother Herbert Powell hears about the lottery win. Boy has he got the mother of all investment options for Homer – nuclear powered cars!
Adjust down payment
Homer can get in on the action if he puts some cash into building a prototype. So Homer has to syphon off yet more funds from the deposit he’ll make on his investment property, and change the paperwork again.
Whew, put all that through the system and see where you get to. If you think of more variables as you go, you can add them to the scenario and run the test again.
What We’ve Tested
A whole heap of stuff.
We tested rejections, with Homer’s first application, and signature recognition when he goofed up his name.
We tested multiple applications made by the same person, with an adjustment in the deposit amount made after the application had gone through.
We tested how to register multiple assets with different mortgage amounts, and a different percentage of ownership. What’s more, the owners of the properties and mortgage were not residents at the same address.
The applicants had different credit ratings, which affected the different algorithms in their application process. And they weren’t related, and didn’t intend living together at the property, which was for investment only.
Here’s a snappy list:
Rejections
Editing documents
Multiple applications from the same person
Adjusting deposits
Multiple assets with different mortgages
Different percentage ownership
Different credit ratings
Unrelated co-owners
Investment property applications
It only takes a little imagination to try to find many more bugs using a soap opera scenario, versus the standard “works as expected” response we’d have gotten from the test-case walk through.
Here’s a three minute recap in a lightning presentation I gave at the Selenium Conference in India.
I recently asked myself, “where would I like to take my career?”. This is a blog post on that reflection. From maintaining tech skills to improving my coaching. Here are the things that are important to me.
Keeping tech skills up to date
I’ve been doing software testing now for over 7 years and even though I come from a Computer Science and Engineering background, I’m finding people too easily put me in the box of “non technical manual tester” based on my previous roles. If I do not work hard at maintaining my tech credibility I see a risk that I will be stuck with the “non technical” label for the foreseeable future. So how am I maintaining this skill? I’m working towards the following:
working towards a certified ethical hacker
Building a mobile app as a side project
Attending developer focused conferences
Certified Ethical Hacker
I’m working through Troy Hunt’s and Dale Meredith’s, certified ethical hacking course on Pluralsight. I find security testing more interesting than automation testing and there’s a huge demand for technical cyber security skills. The course is hard to do, there’s 80 hours worth of lecture material and it requires a commitment to regular learning to maintain it.
Wearable app side project
I’ve already spoken at conferences about my example poo tracking wearable app to talk about ideas like privacy by design. I haven’t even built the app yet and I’m using it to teach people. I’m interested in teaching myself mobile app development and how to build basic API’s. I even have someone who’s offered to help build this. Eventually I’d like to use this app to help teach people about different elements of testing.
Attending developer conferences
I’ve always enjoyed attending dev focused conferences. I always learn more compared to software testing focused conferences because it is out of my domain of expertise. I will continue to prioritise these conferences over software testing conferences this year.
Any new role that I start has to help me with maintaining my tech skills first and foremost. This is number one priority I will look at when assessing new roles.
Improve my Teaching/Coaching
I have always loved teaching. I was the kid in school that all my friends asked for help with maths questions. I’ve been tutoring my peers for as long as I remember. I’ve had my tutoring side business for almost as long as I’ve been testing. If teaching wasn’t see as a bit of a dead end low respected job, I would have studied it at uni. I do this stuff for free, that’s how much I love doing this. Being a Quality Coach at Campaign Monitor showed me that there is demand for people to teach software testing skills. In a few years I’d love to be running my company where teaching technical testing skills is my focus. I’ve been running robotics workshops for kids since my uni days. Any job that allows me to practice on the company dollar is a plus in my mind.
Speaking at Conferences
I’ve spoken at 14 tech conferences in the last 3 years. I enjoy getting up on stage. Any role that encourages me to do more of this is awesome in my books. It relates to the enjoying teaching, however getting up on stage is a little different to tutoring/coaching a person one on one. There is an element of entertainment with getting up on stage. The putting on a persona and pretending to be something more that what you feel inside.
All the data
In all of my previous roles, I’ve always enjoyed doing a deep dive into data analytics. From creating dashboard for event tracking testing to helping product understand how our customers were using our apps. I enjoy it all. I’m considering doing a Masters in Statistics in a few years when I’m more on top of my finances.
A bit of job stability
I went through the motions of job hunting 3 times last year. I do not want to do this again and again. Also meeting new people is mentally draining. A bit of stability with my work environment would be awesome. Or at least some having planned turbulence. I hadn’t had much luck with full time positions last year. Maybe this year I do all contracting? I’m still considering full time though.
Have a good culture
I care deeply about having a good social connection with the team I’m working with. I need to feel like I can bring an authentic version of myself to work. That means being vulnerable enough to talk about my struggles with depression and to be comfortable bring my quirkiness to work. It comes down to psychological safety.
Reasonable salary
I’m not solely motivated by money but having a consistent budget compared to my previous role is important for my financial security. I do feel like I’m getting at the expensive end for being a software tester and I can’t even compete with off shore testers in regard to pricing.
Practising a bit of sales and marketing
I struggle with selling testing as a craft where you want skilled people doing it for you. I’m glad I’ve never had to pitch my skills to CEO’s before getting hired as a tester. Someone else has already been convinced of the value testers bring. However with shrinking test teams I think the testers that will thrive in the future will have their pitch just perfect. Any testers who knows how they add value to others and remains relevant probably won’t struggle as much as those testers who don’t. I’m practising marketing through things like my blog and running Sydney Testers. Getting up on stage is related to marketing too. I’m also going to right a book this year as a marketing exercise.
I’m not interested in
Management; this feels like a clear way to let you tech skills go rusty and it’s becoming harder for managers to promote themselves as relevant in today’s agile/self driving teams.
Having QA in my title and working in a role that is only “Automation” testing; I find a lot of companies are demanding QA automation skills but don’t seem to understand the value a skilled tester can bring to the role or company. QA often stands for quality assurance, however building automation doesn’t assure quality, it helps the engineering team deliver code with higher confidence. To me QA stands for Quality Advocate. I can’t assure quality. You will find I never call myself QA anything in all of my online profiles.
It’s rare that I find someone who can clearly articulate why they want automation in a way that drives me towards it. I’ll happily collaborate with an Engineering team to help build this however I have a tendancy to get distracted by all of the other fascinating elements of quality. Like accessibility or security.
I’d prefer to call myself a Software Engineer in Test who is focusing on helping developers own and improve their own quality processes. There shouldn’t be a difference between Software Engineers and Testers in today’s agile teams. We are all working towards building quality products.
Summary
I’ve come to believe that my next ideal role is some kind of developer advocate role. Tech will be the focus, but that blend of marketing, community engagement, technical writing and speaking at conferences is appealing to me. Ideally I’d like to have a Mobile Software Engineer in Test role to help me move towards my coaching/teaching goals and to help me keep up to date with mobile technology.
What are you looking for in your career? Do you have a career coach to help you move in the right direction?
2018 has come and gone. You know what, overall it’s been a pretty shit year for me but the goal of this blog post is to take a moment to practice gratefulness and to reflect.
The crappy bits
I started 2018 not being able to walk because I broke my ankle in December 2017 and couldn’t walk for 12 weeks. I was also starting a new job. 2018 saw me go through a relapse of depression because of broken leg blues and I went through 2 jobs. The first one wasn’t the right fit and the second one had cash flow problems. But still, these were unplanned events that made me feel like failure.
However that is enough moping about. What else did I achieve in 2018?
India
I spoke at the Selenium Conf in India in July. This was pretty cool. It doesn’t matter how much people tell you about cultural differences, it’s really worth experiencing some of these things yourself. I had never been to India before so that was exciting.
Other Conferences
I spoke at a few more in Aus (Agile Australia, and Australian Testing days), you can see all of my recorded presentations here. I also attended a few offering my sketch noting services. I enjoyed being able to attend these conferences and add some value back.
Public Transport
Getting around on wheels made me appreciate public transport here in Sydney. Sure it’s not 100% perfect, but it’s definitely better than New York. Watch Zach Anner on his quest for the rainbow bagel using the New York public transport:
Sydney Testers
The community here continues to amaze me in the quality of events we’ve done. We’ve really had a good year considering all of the ups and downs the committee has gone through. Here’s a recap of nearly 40 events done in the last year.
We’ve had international guest speakers; James Bach and Michael Bolton thanks to Catherine Karena’s training efforts. There’s been events from Security at Qantas to Mobile Automation at Woolworths. Thanks to everyone who made this an awesome year for the community.
Thanks
Thank you for being a part of my year. Here’s to making the next one even better. This one I will get to start it on the right foot (*tehehe*, pun intended). Here are some goals I’ve set myself for this year.
Last week I attended the YOW! 2018 Sydney Conference. I enjoyed the experience. I was able to host a track on the Friday, meet some awesome speakers and had taken some awesome sketch notes. Here are some selfies with the speakers as I gave them their sketch notes:
The premise of this talk by Kevlin Henney (go check out his blog on Medium) was that everything that is considered “New” is actually quite old. But this reminds me of a quote from CP Grey;
We think of technological change as the fancy new expensive stuff, but the real change comes from last decade’s stuff getting cheaper and faster
This talk was about how diversity, purpose and organisational culture can help teams achieve great things using examples from the code breakers of world war II, Skunk works jet building projects and Xerox Park. Randy also gave this other talk at YOW on engineering yourself:
Cloud Performance and root cause Analyst at Netflix by Brendan Gregg
This talk was a summary of all of the different tools used when doing a deep dive investigation of performance of the many micro services that Netflix uses. Similar vein to this talk:
Cloud, Containers & Kubernetes by Bridget Kromhout
Bridget had once spoken at 30 conferences in one year. Now she averages a more reasonable 2 per month on average. Containers will not fix your broken culture is an interesting follow up blog to read. It was along a similar vein of this talk:
Events and Commands for Asynchronous Microservices by Chris Richardson
Future of High Speed Transportation by Anita Sangupta
No Code by Advi Grimm
Advi asks, do you really need code to help you build that solution?
Shaving the golden Yak by Jessica Kerr
Jessica also gave a keynote on the overlap of opera and programming
The problem with pre-aggregated metrics by Christine Yen
Check out this interview with Christine Yen:
Top ten security flaws by Gary McGraw
Similar in vein to this video:
Conclusion
YOW! was a lot of fun with many highly regarded international speakers who write the books on these things. There were themes of an old analogy is seen as “new” (opera and programming, 1968, and breaking codes). There’s many new tools out there to keep an eye on in the near future.
I’m starting my goal setting early. I wrote this blog for goal setting for 2018. On reflection; I haven’t slipped back into obesity at least, which I was super concerned about as I spent the first six months of 2018 recovering from a broken ankle. Here’s my thinking behind my goal setting for 2019 and why I’m starting early.
Brainstorm everything I want to do
As an exercise, I listed everything I want to do and then asked myself, What do I have time to do? What is more important? What aligns the most with my personal values? I’ve had to eliminate a lot of extra curricular ideas and I still feel like I have a lot on my plate 🙁 .
I don’t have enough time to do all the things I’d like to do
I then came up with the following list of things that are really important to me. They are themed around personal, career, family and financial goals:
Beat the overweight label (personal)
Maintain a daily meditation practice
Write a book (career)
Teach my Nan digital marketing (career and family)
Launch an app
Pay off half of my credit card debt (financial)
Speak at one international conference and take my Mum
Brew two whole grain beers
Keep Sydney Testers going
Create enough content to run a 3 day workshop
Why start the goal setting early?
I’m going to focus on developing my morning habit for the rest of this year. I’m going to get up early, meditate and write before heading off to work. If I can do this for the rest of the year, I’ll be in a good place to expand it come 2019. The green in the following table is this minimum commitment:
I’ve put together an idea of what my ideal morning and ideal week looks like. If I do not put aside time to do things that are important to me, it’ll never get done. I reflected on what I could squeeze in. Unfortunately things like studying Japanese just don’t fit in financially and time wise. So here’s my ideal week to work towards in 2019:
There are some goals that I haven’t put aside any weekly or morning time for but they can’t easily be chipped away with a daily/weekly habit.
Keep goals measurable but hard to achieve
Everyone seems to be talking about Objectives and Key Results (OKRs) for goal setting these days. One of the important things with OKRs is that they are hard to achieve. When you reflect back on your goals you should be able to say you hit up to 80% of your objective. If you hit 100% you actually set your goals to easy. For example, I’m going to work towards beating the overweight label but I’m not going to consider myself a failure if I only get halfway there. By listing up my ideal morning/week it gives me an ideal to work towards but by highlighting my minimum commitments I won’t beat myself up if I have a bad week or two.
External Accountability
There’s no point in setting vague goals that you don’t tell anyone about. To ensure external accountability with my goals I’m going to;
Pay for a personal trainer for a twice a week weight lifting session
Go climbing with my partner every Wednesday and Saturday
Pay for a publisher’s time to help keep me focused on writing
How will you go about goal setting for next year? What measures will you take to ensure accountability? Please let me know.
I’m a visual learner. I like to draw things as I absorb information. I enjoy doing sketchnotes while I’m at a conference. See YOW! Sydney 2018, Australian Testing days and Agile Australia as examples of these sketch notes in practice. It helps keep me in the moment and focused on the talk material. It’s also a nice thing to hand to the speakers as they get off the stage. Here is my workshop material for learning sketch noting:
Everyone can draw
You brain is a pattern recognition machine and will turn almost anything into something you recognise. Even your random squiggles can turn into birds. Try this squiggle birds exercise out as a warm up:
Mindmapping
Mind mapping is a good way to start with visual thinking. You have your central idea in the centre of the page and all of your ideas related to that idea radiating out if it:
You can use a mindmap to brainstorm ideas like, “How do I test a username/password field?”. Michael Bolton has a this fabulous mindmap just for this problem:
A Michael Bolton mindmap
Sketchnotes
you can do sketch noting without drawing. You can start with lettering and things like bullets, frames and connectors:
I use banners everywhere
You can even find youtube videos for doing these banners
Build up a library of icons
There are many common icons you’ll use. I often draw light bulbs, locks, poop emoji’s and tools (what does that say about the state of technology?).
Use colour to highlight ideas
It might feel like you are back in primary school colouring in borders but I love adding shadows and some colour highlights to my sketchnotes to really make them stand out/seem more 3D.
Practice your stick figures
people are often used to communicate abstract ideas. There’s lots of different styles out there and you will find your own.
Give it a go
The next time you are watching a lecture/presentation on youtube, try and take some sketch notes and let me know how you go.
I had a blast at Australian Testing Days Conference in Sydney on Friday. It’s always good to get reconnected with some colleagues and to make new connections. First up we had Jennie Naylor go over how to use OKR’s to drive the Quality Onwership:
The OKR (Objectives and Key Results) framework has been in use since the 70’s and is used by companies such as Google, Uber and Facebook to set their organisational goals. OKR is a framework that can be used at any level within your organisation. This presentation mention how to use OKRs to help improve quality and ownership in a team.
Key Takeaways
The key takeways I got from this talk was how to use OKRs to determine if experiments are successful or not and getting the whole team involved in quality and product ownership.
AI and the impact on QA
Given by Chhavi Raj Dosaj who proposed the question, can AI can the landscape of testing? Chhavi had a big topic to fill. I think his main point was a little lost because he spent too much time going over the basics and he wasn’t able provide compelling examples of where AI could benefit the QA process.
One of his examples was using an unsupervised learning algorithm to select the top manual test cases to execute in the next testing cycle based on past performance of said test cases. Anyone who primarily uses test cases to manage their testing work and reporting is using a pretty dated testing practice that doesn’t scale or make sense in the Agile world. You can read more about my thoughts on the end of manual testing. I also wish he asked me for feedback on his slide design before he got up on stage. There were some presentation sins committed in his slide deck.
dev Ops Testing Strategy
DevOps Testing strategy by Amit Kulkarni fell victim to the old bait and switch presentation sin. The title didn’t match up to the content. My key take away from this talk was, “there are tons of tools out there you can use in a continuous testing way”. There wasn’t any DevOps in this talk at all.
Performance Testing in CI
Andrey Pokhilko gave an engaging talk on Performance Testing in CI using opensource tools. I learnt the name of a new tool; Taurus. Which can be used on top of JMetre and is a little easier to use than the good old JMetre.
State of BDD
As I’ve already done Gojko Adzic’s spec by example workshop offered through YOW! a few years ago, I already had a decent idea where this talk would go. Bria Grangard was engaging and gave a good update on the state of this field.
Transforming Testing
Bruce McLeod gave an engaging talk in business strategy and how it will transform testing. This was one of my highlights of the conference. Bruce goes over how testing needs to change from a culture of “protection” to a culture of experimentation using the Netflix chaos monkey as an example.
