Risk Based Testing | Part Two | Uncovering risks

Welcome to part two of this three part series on the relationship between Testing and Product Risks. 

In part one, I discussed the idea and benefits of talking about testing product risks instead of talking about types of testing. If you missed that post, you can view it here.

In this second part, I’ll talk about how to uncover product risks, and share a model that I use that helps me flow my thinking in uncovering variables of the things that I test, and then variances of those uncovered variables too. All before we then ask the question on whether each risk we have uncovered actually matters.

How to uncover product risks

Here’s a model that I use to describe how I approach identifying risks in the thing I’m testing (or going to be testing, if I am starting to test the idea of the thing):

If we start with “the thing”, then we can break this down into different parts. Not just the tangible things but the intangible things too. We can think about things like the purposes of the thing, the properties of the thing, the users of the thing, each different integrated parts of the thing, the layers of the thing, the states of the thing, etc…

Then we can think about variables of each of those different parts: Multiple different purposes and properties, multiple different integrations and layers, multiple different states, etc.

An example, using my son’s toy piano…

If I use the example of my son, Angus’ toy piano (the pic above shows what the toy piano looks like), the primary purpose of this would be for Angus to have fun bashing some notes randomly with his little fingers (and when he’s old enough, to perhaps play a tune?). 

I also like to play it too. Although it’s small, I can still play some chords and use both hands to add some harmony notes to some of the nursery rhymes that I know how to play. So equally, this is another purpose of the toy piano – me playing songs to Angus. Also, with Angus being at toddling age, he likes to try to climb on it and use it to climb up on the couch (as much as I don’t like him doing that for his safety), or sometimes he finds it entertaining to turn it on and off a few times very quickly, as when you turn it on it plays a little jingle. Also, he finds the switch really enticing to press too 

We can break down its various properties: keys (black and white ones), a wooden frame consisting of 11 wooden panels (that i can see), a speaker on the base, the on/off switch, batteries, an internal circuit board, patterns painted on it, etc, etc, etc.

There are also various states we can think of relating to this toy piano: the on and off states of the product, Angus has played this while sitting on a chair, while sitting on the floor, while standing, he’s played it with his toes and his elbows, he’s also placed the piano on its back to play it like that. I’ve played it while it’s been balanced on my knees, I’ve played it while Angus has been leaning on it too. We’ve played it at different times of day. We’ve played it when the batteries were running out, and then again immediately after I replaced the batteries.

You get the gist – there are lots of different variables here for the toy piano. And each of those variances have their own variables too. I really have but listed just a few of these variables in my example and there are far more unknowns that I haven’t listed.

Thinking in this detail across these different levels allows us to generate lots more ideas about many different product risks.

And here’s something funny: when I play the you piano, I hear various bugs from it. When playing certain keys together, it randomly adds odd notes… But when Angus mashes around on it, he never causes the piano make the same problem noises, since he’s playing single notes and not chords. I’m not even sure I’d notice if he did make the odd note sounds, as the notes he plays are all a bit random anyway!

To feed your curiosity regarding what this odd note sounds like when playing 3 notes simultaneously, here is a link to a quick video so you can hear it (note: I've put this on my OneDrive, as LinkedIn doesnt allow video uploads):

“DOES THIS RISK MATTER?”

After we’ve thought about different potential risks, it’s important to ask the question: “does this risk matter?”. This is an important question simply because the risk might not matter at all, so you really shouldn’t waste your time. We certainly can’t test everything, so we need to focus our efforts on the risks that do matter. Prioritizing our testing is important.

BUT! (There’s always a but…) Deciding whether a risk matters or not should be collaborative between you and your team, and a product representative. You might not think it matters, but you might not have considered a different perspective that someone else in the team is aware of.

If you reach a consensus decision that the risk doesn’t matter, then that’s when you should make a note that the risk is being accepted and move on. The note is important, because things might change. The risk may become important in the future.

Your testing notes tell the story about your testing, test ideas, decisions and other relevant info. And if an unimportant risk does become important in the future, the story: “from our testing notes, I can see that we identified and discussed that risk, and deemed it unimportant for these reasons…” is a very different story than “um… It’s not in our testing notes, but we did speak about that, I remember we did accept the risk, but I didn’t take note why.”

In the final post of this three part series, I’ll discuss the perspectives of testing to uncover risks, and testing the risks to discover problems – the possibility of mitigating risks through design. And I’ll share a few more models too.