Mob Testing —why not crowdsource your QA?

The other day I was reading a software development subreddit in bed (my hobbies are dangerously intermingled with my professional life, I know) and there was an interesting thread asking: ‘how do you test the things you build?’. The top answer was a tongue-in-cheek response of: ‘our end users are the testers’. Of course, I don’t agree with this for several reasons (the main one being I wouldn’t currently have a job if everyone did this), but it did get me thinking; no amount of testing really prepares you for how an end user is going to use your systems. Of course, due to human error and even human ingenuity, there are always going to be bugs or dodgy user journeys that make it into a production environment. Why not expose your latest feature to as many people in the business as possible before putting your changes live? This is the concept of ‘mob testing’ and it has proven to be very valuable for my team!

The fundamental point of mob testing is exactly that — forming a ‘mob’ of colleagues in order to get as many eyes on a new feature/system as possible before a real end-user ever gets to see it. You can condense dozens, if not hundreds of people hours into a couple of afternoons and catch a load of bugs before they ever see the light of day. One of the outstanding positives of this approach is the wide amount of modalities and backgrounds you can add to your mob. The mob doesn’t have to consist solely of purely technical people (although they are useful to have); chuck in a few business analysts, product owners, agile leads, hell, even let the dog visiting the office have a stamp on the keyboard and see what happens. Having a breadth of differing experiences and opinions can lead to some really interesting finds

.How you conduct mob testing sessions is entirely up to you — I have found some success in doing a ‘hot seat’ approach, where you have one ‘driver’ and everyone else watches them interact with the application, gives directions and keeps an eye out for any errors or bugs cropping up. You could also split into smaller groups, or let people go off by themselves and report back with anything they find whilst using the application within a blocked out timeframe. One very useful part of working in a larger group is that you can appoint a member of the group to take more detailed notes about the bugs you find. Ensuring that you have a high quality, verbose test report at the end of the process will lead to quicker fixes.

The biggest barriers I have encountered to getting a mob test set up is giving people enough of a heads up to get involved and also encouraging people to get over the ‘fear of testing’. A lot of people are scared to test things (sometimes rightfully so) or become reluctant to get involved in things because they seem dauntingly technical. The former can be addressed by making sure a mob test is only going to take place towards the end of an iteration of the product or when you know it is imminently user-ready. If you roughly know this information, you can start making noise about it on your channels of communication well in advance; making a private channel people can join with all the timings goes a long way to help fill out your mob.

To get people to want to be involved, you have to break down the barrier of technicality. Write clear instructions on how to get the feature set up, or better yet, make a quick follow-along video that anyone in the business could potentially watch and get started. Ideally, the mob should be interacting with the application in an identical fashion to your end users and there should be little to no knowledge of how the underlying code works necessary to be able to test effectively.

Of course, like everything else in software, mob testing isn’t a perfect solution. Due to the nature of the product/iteration needing to be basically finished, finding any bugs at this late stage may hinder any planned releases, or could even lead to some rewriting of code very late in the development cycle. This is, of course, better than an actual user discovering the bug. Mob testing could also be seen as an excuse to be more lax with testing further left (earlier) in the process; it could also lead to more bloat in higher-level tests such as end-to-end tests to mitigate any regression found whilst mob testing. There is never one true path when it comes to testing — finding the balance across all methodologies is part of the fun!

My team have personally found a lot of value in carrying out mob tests on the last few releases we have done, both before code went live and for hot-fixes. A crazy amount of edge-case bugs have been found and squashed before they ever became a real problem. Quality isn’t just the concern of the ‘testers’, as everyone in the team is responsible for quality and mob testing is another tool to make that ideal more of a reality.