When the bug is 'non-reproducible'...

Murphy’s Law states - “Anything that can go wrong, will go wrong!” - but do you know how it’s going wrong? What if it doesn’t go wrong again? What if you can’t make it go wrong again?

You find a bug, head over to the dev team and you say  - “This is a bug. This is not the ideal user experience. Are we okay with this behaviour? This is not as per the Acceptance criteria. Why has this stopped displaying/working suddenly, it was working sometime earlier on the same build!?”

And the reaction you get is - “I can’t recreate it, so all is well with the product!”

But what do you do when customers report it!? 

Always remember, the real-life navigation often differs from what we test.

Well, it’s easy to say “it’s a bug” and hard to document it or even recreate sometimes for yourself and for Devs at other times. 

When Testers try experimenting with the product/app, we end up imagining so many scenarios. 

In the current era of digitalisation and when everything is app-driven, the user has ‘n’ number of ways to use the app and deal with that functionality which you thought would be used just the one way.

Here are some unique situations and reasons why you/ Dev can’t recreate the bug (and I am not speaking about flaky tests here): 

1. A bug can be related to a particular default language the device is set to. The device language and the language which the app supports (depending on the legal & region-specific guidelines if any)or displayed in could be different to avoid unwanted errors. 
2. A bug could be limited to the OS version of the device. The 3rd party library may not support the old OS version, e.g. iOS 12, though the app itself does. OR a few iPhones are still with iOS 12 and not updating to iOS13/14.
3. A bug could not be an app issue but something with the analytics library the app is using, and that library could be crashing the app abruptly when you background or lock the phone or switch between the apps.
4. A bug could occur from an earlier happy scenario you performed but then end up in error for the next scenario - this could be because the cached data was not updated. Or the user deleted the app cache or browser cache/data that the app expected to have in place!.
5. A bug/error could be because you had that camera app open while trying to use the camera from the app.
6. A bug might display the currency in the region selected like INR instead of pounds, and it may end up in the wrong display of debit/credit to the user.
7. A bug could be because the device exhausted the RAM with so many apps open in the background and the user is seeking to multitask by switching between the apps (copy-paste, referring to another app for info, etc).
8. You are trying to access certain data/links for some transactions and you notice blank account numbers or records. Was there a network off-on situation that the app didn’t handle?
9. A bug could be because you didn’t let that “pull to refresh” complete. And as Devs are often running the app in simulators or with the stub/mocks, the refresh issues never actually affected them.
10. A bug could be intermittent. It may appear after 3 times you cancel the operation and the app hangs! Or on a success page, you double click the back button and boom!

How do we help ourselves and devs to re-create a bug then?

• You should relive the exact steps you followed. Clear cut steps are one of the most probable ways to reproduce a bug.
• Have logs information & attach the video of the steps you followed, a screenshot of the error screen will be a plus.
• What if you don’t remember the steps? Check the logs and it should give you the exact thing you did with the exact scenario and the exact error. Check my article to learn how Logs are BFF to Testers.
• When you log the bug - make sure you mention the specific device, specific OS version, the browser and its version, Network to understand if it’s a general bug or specific to some kind.
• The Precondition - state of the device, app, the functionality performed just before the bug showed up and the data - is one of the critical factors. We often forget to note these down. 
• Try the same steps on different browsers, devices, iOS, Android, with different OS versions to see if the bug appears there as well.
• Check if you had some specific user or a user with some odd combination of accounts/carts.

With just a little extra awareness, the non-reproducible bug could become a potential bug under certain user/device conditions. 

Do experiment, but by keeping track of the steps/logs, an eye for the risk/problem would help save the revenue and prevent those bad ratings for the app.