Upgrading your Testing Strategy
Nikolaj Tolka?iov
Engineering Manager @ NordPass at Nord Security | Co-host @ “Pokalbiai Apie Kokyb?” Podcast | Executive Board Member @ LTSTQB
When was the last time the testing strategy captured your attention?
Below is the result of the quest to revise the testing strategy to ensure that clients, stakeholders, and the team involved in the projects could quickly grasp the most crucial highlights, track progress and easily locate the information they needed. This ended up as a stark alternative to a lengthy 12-page document we used for testing strategy.
To be fair, implementing an extensive testing strategy can be justified, particularly in highly regulated industries. However, I've observed that these comprehensive documents often suffer from a lack of updates and engagement. Teams tend to avoid revising them, and high-level managers, who are ultimately responsible for financing the project, may not have the time to thoroughly review such lengthy document.
Introduction
This is not a complete guide how to build testing strategy from scratch. This is a quick overview of visually appealing presentation of testing strategy with the goal to present key testing strategy elements in the condensed and easy to digest manner. You can use this as a building blocks to facelift and present your testing strategy.
This modern testing strategy is comprised of three distinct components:
Detailed view
Let's start with the detailed view and use bottom-up approach to get familiarised with this modern testing strategy. Each category is fundamentally structured around a scoring system (a grey bar in the picture above), focusing on the extent of best practices and quality-related activities improving that specific testing category. We will touch upon scoring system later in this article, but gist of it is that you have a list of activities/processes/goals and completing any of them improves testing strategy score.
If you lack budget to support investment required to perform security testing, it will be reflected with a lower score. This scoring system ensures immediate visibility, highlighting which testing area in need of enhancement.
What category score can tell us
Category detailed view is not an achievement list but rather a snapshot of current state of testing strategy execution.
Let's imagine an "unlikely" scenario where the team becomes overwhelmed with tasks and struggles to maintain pace, leading to insufficient time for writing test automation for new features. This should be represented in the scoring system and crucial that this situation is made transparent to all stakeholders. This can be used as a way to escalate deteriorating product's quality gates which might impact end users.
In the picture above you can see test automation category with it's list, in this specific case our goal was to enhance test automation by incorporating backend and frontend unit and integration tests into test automation strategy. Additionally, we included nice-to-have elements into this category scoring system, like:
Components of the category score
We will expand on categories latter, but you can have various categories, like test automation, accessibility, security, etc. For each category, you can add items that would enhance your testing strategy and end up as a part of category scoring system.
To simplify the process of creating the strategy from scratch we maintained a categorized database to facilitate the selection of the most valuable items for each category (example above shows accessibility items and scoring). That database would be maintained and used by multiple project teams. For each project, it was necessary to select only relevant components. For instance, if the project does not include a frontend and focuses solely on backend testing, adding UI end-to-end tests in the scoring system would not benefit the strategy.
How to calculate the score
Now, let us proceed to formulating the scoring rules. You can use 0% - 100%, A-F, or any other scale. I refrained from using percentages because they imply a sense of completeness, which I believe we cannot achieve at any stage. The A-F grading system assessing the current state and configuration, rather than on evaluating completeness. You can distribute needed amount of completed items to qualify for the next tier equally or make it less demanding in the beginning and more at the higher tier. For example:
Notes section
The notes section can be used to include important details which otherwise do not belong in the scoring system.
领英推荐
Agreement on the score team will strive to achieve
Progress since the last update
Prioritizing items on the roadmap
Using visual cues for a quick status overview
Test Automation Strategy Overview
A one-page overview of all categories, it can be a first or last page of the testing strategy presentation. In the top part you can see an overall score adjacent to the "Summary" section (in this case it's C+) this is an average of all category scores in this testing strategy.
Most of the page real estate is occupied by categories and it's scores are expressed as a horizontal bar chart. This view helps us see gaps and compare them. Each category have a detailed view which we discussed in the previous section and their individual score can be analysed if necessary. In this example accessibility testing is not applicable (N/A), as application was intended for internal use only and would not be available to the general public. This can be included or excluded from the list depending on the need to align the expectations like what type of testing will not be in the scope. This condensed overview is practical for initiating discussions and aligning on priorities.
All the categories in the example above should be self-explanatory, but let us address the elephant in the room - Definition of Done. This one is for sure an odd one in the testing strategy and might have even raised your eyebrows ?? The rationale behind this is to push test automation as a part of DoD, so we would have all necessary components to avoid having manual regression testing. In your case, you might want to exclude this category from the testing strategy.
Real life use case
In one of the periodic demonstrations, we revealed that our performance testing was rated at a [F] grade, and due to budgetary and resource constraints, we were unable to improve our performance testing score. Despite this disclosure, stakeholders chose not to invest in improving performance testing. Subsequently, a significant performance degradation occurred after a few more releases, leading to a rollback in the production environment. Proactive and transparent communication with stakeholders about the shortcomings in our performance testing helped us build trust and ensuring alignment of expectations.
Testing Strategy Appendix
The final section of the modern testing strategy is the appendix, which comprises several free-form pages covering various topics such as the testing process, release process, quality gates, defect severity levels, and more.
Final Remarks
I hope this article has sparked a few ideas on how you can integrate aspects of this modern testing strategy into your workflow, focusing on alignment and reporting. I recommend extracting aspects from this testing strategy that would improve your current flow and avoid doing a complete overhaul.
Custom software development firms can leverage this approach for evaluating potential "brownfield" projects or during the Request for Proposal (RFP) process. This method will demonstrate your consulting prowess in an acceptable format for decision-makers and help justify budget expenses.
If you have any additional questions or could use further clarification on certain aspects, please feel free to reach out to me. I am open to and appreciate any feedback and will use it to enhance and refine this article.
Big shout out to everyone who commented on the LinkedIn post and expressed interest in learning more about this testing strategy. Your enthusiasm and curiosity have inspired me to share this knowledge. In this sense, you are all co-authors of this shared learning journey. Thank you!
Special thank you to Vaidas Kavaliunas and Justas Lau?adis for peer review and feedback. Your feedback helped improve this article by a lot and it would not be as good without your help.
Helping risk management & leveraging quality ?? Engineering Manager @Netguru
9 个月I tend to start the strategy with knowing who is interested, how are they interested/how they define success and finally how are they gonna assess that things are moving closer to this definition of success.