The importance of mindset in testing

A while back a colleague of mine was unsatisfied, when I told him, that a lot of what we do as testers is about mindset. I don't blame him, because talking about the importance of mindset without context and concrete examples is vague at best. So let's get to context and examples then, shall we.

As many of you know, at SOK we are building a web service, where you can order groceries and have them delivered to your door or pick-up box. Yes, I'm talking about the legendary S-kaupat (s-kaupat.fi). One feature of this service is that all alcohol purchases should be made between 9:00 and 21:00 and it shouldn't be possible to purchase alcohol outside this range. Let's see how three different mindset caricatures would approach this challenge from a testing perspective.

How would a developer test it?

The majority of developers (not all, of course) I've worked with approach the problem by implementing time-related restrictions on it and few automated checks on unit, integration and/or UI level to make sure that the feature has been implemented correctly and doesn't break/regress when changes occur. All good and very professional.

How would a QA engineer or SDET test it?

The majority of testers, QA engineers, SDETs and such (not all, of course) approach this by going deeper into it. Enriching the suite. They try to find the bugs that pester the feature and the people dependent on it. Many of them are classically trained, which means that they understand testing techniques such as boundary value analysis and equivalence partitioning. They test values such as 8:59, 9:00, 9:01, 20:59, 21:00 and 21:01 and perhaps update automated test suites accordingly so, that these tests run every time a pipeline moves new build into test/staging environment or even production. More experienced ones might suggest that external factors such as daylight saving time might cause problems. Test suites are updated and automation is set to run also periodically to catch the external factors. All good and very professional.

How would Sami test it?

I'd make sure that everything I have is called into service to find out, if it's possible to introduce a scenario where alcohol can be bought outside the range between 9:00 and 21:00. There are numerous time factors, contractual and legal boundaries, interpretations, dependencies, configurations and data, that can be manipulated. More testing techniques. Heuristics. Some personal and some well matured in testing communities. Facilitation models, so that others can join too and harness their testing power for this endeavour. And tooling of course. But it's not random. It's highly structured and often timeboxed, logged, purposeful and focused. For me every test that doesn't introduce this scenario is ultimately a failure, but it may reveal something that eventually leads me to this scenario. Or does not and we learn something else. Another test idea. Another risk. Another way to look at our product/service and another way to study it.

Sami, isn't this an overkill?

"If you don't have time to do it well, you don't have time to do it twice." -Proverb

Good testing takes effort, for sure, but actually way less than trying to milk the thing via production and do all the rework. One might think that customers and end users find these kind of bugs, and they of course do eventually, but there's one problem in that plan: In this context it's illegal. In Finland it is against the law to sell alcohol from grocery store before 9:00 and after 21:00. If you make the decision of letting customers and end users find these kind of bugs, you'd be committing a crime.

Legal factors are not the only thing that you should consider when tempted on having your products and services tested by customers and end users. This temptation impacts company image, costs, rework, churn, team morale and root cause analysis, just to name a few. Of course a lot can and should be done with the help of the masses, but having that as the only approach brings you on thin ice.

Can these mindsets be combined?

All these mindsets can of course be combined and should, to some extent. But I have a hard time understanding how anything good could come out of putting all this on the weary shoulders of one person. Or a homogenic group of people trying really hard to make something happen and at the same time trying to come up with scenarios that bring doubt on this. During my almost 20 years of travels within 20+ different business domains, 40+ projects and countless teams I have never seen that happen in a sustainable fashion. I've of course met brilliant people, devs, designers, POs, SREs, QA engineers, SDETs, who know all this and master it, but when the going gets tough, and in these days it usually does all the time, compromises happen and people start shifting their mindsets to the basics. Whatever that may be. At that point you need someone in the team, whose basics are within testing and the uncompromising mindset it requires. No matter what the official title is, I shall call this person, deservedly so, a tester. :)

It's a team effort

Now, we have all the pairing and mobbing techniques, retros, blameless postmortems and such and they help tremendously in putting all these mindsets into a team, but at the end of the day the success comes from having people in the team with different mindsets. Personally I believe that diversity doesn't come from race, gender, cultural backgrounds or even competence, but having different mindsets within a team. Combining these mindsets and having open, honest collaboration between them is the key to success, and quality.

There. A context and an example. One out of many. I hope you learned something from it.

Happy testing!

PS: I know that there are people reading, who think that testers are just people who doubt things, rock the boat and bring hardships on developers. For lack of a better word, a**holes. In a well functioning team that is quite far from the truth. I have a another post brewing about working as a team and together fighting our mutual enemies: bugs and risks. Let's see, if I manage to push that into production in upcoming years. ;)