Agile Testing: We Do It Wrong
Last year I made a discovery. I realized I had misunderstood testing. I'd thought to test something meant to make sure it worked as expected. Matched the acceptance criteria. Did what it was supposed to be doing.
I was wrong.
If you have ever been to IKEA furniture store you might have seen a robot performing a durability test on a chair. In the middle of the store, inside a glass box, the pressure is applied to the famous Poang model 20 times per minute to simulate a 250 lb person sitting in the chair 20 times per minute.
Impressive, right? I have actually owned this chair for over 10 years and it's still good as new, except for a few scratches coming from my cats.
But if I tried to test this chair using my original definition of a test as "make sure it works", I would probably just sit in it one time and then declare the test successful! After all, it worked for me.
Would it be enough?
See, I am an average size human, maybe on a smaller side, and I sit in chairs in a very graceful way. At least I hope so.
But what about other potential chair owners? Who might be bigger, smaller, who might want to lean, rock, jump, turn, and swing, while drinking tea and chewing on a sandwich? Those who will use the chair not one time, but for many years?
Now, the idea of testing is shifting from "checking that it works" to "finding creative ways to break it before the users do".
Break the system before the users break it.
Software users will put your application to a real test, and they also come in different shapes. Some will try random workflows because they don't know the product very well. The power users will want to dig deeper. And finally, there will be people who will actually want to break into your system with a malicious intent.
So, instead of our typical linear progression from coding to validation, we need to get into a cycle of building the application and breaking it, fixing it and breaking again. We should continue through the cycle of building and breaking until the product becomes resilient.
To break like a user, we need to think like a user.
The real test planning is not about capturing step by step instructions for a test case. It's about creative imagination at work trying to tackle the system from many different angles to find its weaknesses.
Invalid or unexpected input goes a long way in getting insight into what the system will do when the user makes a mistake or intentionally feeds it incorrect data. Testing on a large data set, or a small one, standalone or integrated into multiple systems will widen your horizons and knowledge about the product behavior. Trying to act as different users or user personas might lead to unexpected findings. Putting the system under stress as many users try to access it will help to find the system's limitations.
And if you tested and didn't find any defects, congratulations, your software is perfect! Except, I haven't seen one yet, so it's probably too early to celebrate. It is possible that you are not looking in the right place.
You never know if the system is free of defects, or if they are hiding and you can't find them.
Do not despair. The testing work is rewarding in itself, because it creates knowledge. The more experience you are getting about your product and different ways to break it, the more you learn about users and how they break it, the closer you are getting to making your application indestructible, like that plywood IKEA chair my cats like so much.
Build and share knowledge, learn from users, become better at breaking things.
And most importantly, do it in every sprint. That's the heart of Quality in Agile. We are moving forward so fast, we have to be confident about our product. And the only way to be confident, is to do our best to test it, which means - to break it.
Like and share with your network! Comment, share your thoughts and send me your questions, follow me on LinkedIn and Twitter to read my articles on practical Agile.
My other posts in "Testing in Agile" series:
Product Owner | Digital Report Designer | #Testing | #Migration | Power Bi
4 年I would add that having fun "testing like an Easter eggs harvest" increases the probability to find something - just like a reward! (break the system, find all differences between the resquest & the result but also imagine new creative ideas for later ??)
??Global Director@QA Mentor|??The Go-To Software Testing Partner for Elite C-Suite Visionaries|???? Exposing Hidden Risks Before They Stall Growth ????|?????Strengthening Trust & Resilience
5 年Excellent Article
At my first high-tech job, we had a delightful VP who always found a creative way to break our software a couple of days before we planned to ship it. One of his least-appreciated tricks was simply to launch the software, start a task, and then place a bulky dictionary on the keyboard.
Accessibility and Inclusion Content Manager and Consultant
5 年Love this. I also used Ikea testing as an example recently! You definitely put it more succinctly, though.
Great article, Katy. This aligns perfectly with a comment by an interviewee for my Development Integration group at Broadband Technologies: “Most Software Engineers test to show one way their code works. Test Engineers test to find all the ways it doesn’t work.” I hired her. :) This is also one of the aspects I like about Test-Driven Development: it encourages developers to learn to think like testers.