Smart Testing Ideas

On the way to work this morning I listened to an interview with Christin Wiedemann on TestTalks (for the second time) and picked up some ideas that I hadn’t before. I am always looking for ways to be more effective because I am stretched too thin and am the first to say that I am not able to work to the level that I would prefer (sound familiar?).

Back in my early training I latched on to the Truth Table as my favorite test tool. I list all of the theoretical possible combinations of state, values, and actions with the expected outcome, and then remove any that are either impossible or are obvious duplication. But now that I am the only tester supporting 6 developers, 6 BAs, and a dozen systems,  I just don’t have time to test all these remaining combinations.

The idea that Christin gave me was to ignore the happy path and just test for failures. She reasoned that passing tests don’t teach you anything but failures do. She went on to talk about Likelihood and Impact of a failure as the criteria to prioritize tests.

Just give each condition a score of 1 to 5 for each of these criteria and then rank them by the sum of the two values 

 I realized that if I only tested conditions that had a higher than average sum of these risk criteria that I would still eventually cover the happy path conditions anyway, when these tests passed.

She also noted that when assessing the risk of each condition you need to know which behavior is most important (to the user). Requirements or Stories should have acceptance criteria but it is helpful to ask why these criteria are important because the answer will drill down into the real impact of a possible failure. This understanding can make clear which test cases address the real concerns, most directly.

Christin suggests asking what could cause the failure. What I really want, is to group tests by source of a possible failure. Since one specific failure could fail multiple tests you just need to test one of them to learn about the problem. This works well with another favorite tool of mine, Equivalence Partitioning, which eliminates tests that are testing equivalent conditions.  Often I cannot find or create test data required for all tests. If I know that several tests all address the same potential cause, then I can focus on finding data for just one of them.

With this understanding of how fewer test cases can still cover the most important risks, I feel confident that my testing ensures a high quality deliverable, within the time constraints imposed on me. But I will have to try it and see if it helps catch defects quicker without missing anything significant. The interview discussed ideas from her 2015 Oredev Conference talk.

Thanks Christin and Joe.