Pass the Test: Mark Winspear talks quality
Testing is my passion and I’m proud to lead my global Test & Quality team. In our latest Net Promoter Score survey, we achieved a score of +92. NPS is an industry recognised customer experience metric used to gauge customer loyalty, satisfaction, and enthusiasm on a scale between -100 and +100, by asking the consumer directly about their experience. This is an outstanding result, and a lot of hard work has gone into this by my team, but I have seen throughout my career that testing often is unappreciated and misunderstood and maybe that is because the outcome of successful testing processes is generally that nobody notices, nobody suffers, and people are happy. As a highly respected delivery lead I worked with put it: "You just can't make testing sexy".
In the UK, we’re over 20 years and counting into probably the largest miscarriage of justice in history which has recently been brought to public attention by a TV drama, high-profile inquiry and Government intervention.
When the Post Office commissioned Horizon, a new ￡1bn accounting system for its 14,000 branches, it had serious critical defects in its software, interfaces, and hardware; unrestricted and regular unaudited and invisible remote edit access to production systems and data; and inaccurate and insufficient logging and reporting. When the system reported accounting shortfalls at branches, this was covered up by the Post Office.
?
Postmasters were each told they were the only one complaining about having problems with the system and were blamed for those shortfalls, which they were contractually obliged to repay - often tens or hundreds of thousands of pounds.
Instead of investigating and rectifying errors, the Post Office robustly defended the system in court using private prosecutions.
The outcome of this action is that many postmasters have lost their jobs, homes, and businesses, some forced into bankruptcy and there have been, tragically, several suicides. In this case, innocent until proven guilty could not apply. In the previous manual system, postmasters could reconcile data but now they were simply told they were wrong, and the system was correct.
?
Whilst this article would be a novel if we covered all of the failings (which were not isolated to testing by any means), the points I want you to be left with is that the Post Office outsourced this work to a supplier, Fujitsu, but that did not outsource the risk nor the reputational or financial harm that occurs when risks were realised.
?
When it comes to testing, independent testing is referenced in at least 19 sections across six ISO standards, each with inherent consequences if not in place. In practice this means that however we are using suppliers in our processes, the client, using qualified test professionals, must define and own the testing strategy, process, and assurance processes that the supplier contributes to and follows ensuring that all test processes are visible and there are no unexpected surprises.
?
Remember: the outcome is that testing functions as a risk identification, exploration and reduction exercise with auditable results and process.
领英推荐
Could testing have prevented a lot of these issues?
Testing absolutely should have caught them, but in the modern world, we must do more than just catch defects by preventing them occurring in the first place, and we do this by testing being engaged and involved throughout. The critical thinking of professional testers could have identified issues and gaps in requirements and design upfront regardless of delivery methodology. It could have driven out questions about the operation of the system functionally and non-functionally, improving the requirements and design before infrastructure was built and coding started. With tests designed appropriately, teams can build in quality at the earliest opportunity with the lowest cost.
?
Were there budget constraints meaning there were not enough testers, or a good enough caliber of pro-active and collaborative tester? Were testers isolated from users and therefore did not understand the business processes they should have been testing with them and having at the forefront of their mind at all stages? To be a customer-centric organisation, users must be integrated into the process, not an afterthought. Was there a drive for cheap testing meaning that testing existed at the end of a production line with no (or weak) test leadership?
?
Whilst there are many questions, the honest answer is right now we don't know all the details, but some of these absolutely apply. Professional testers will work with teams on pragmatic solutions to issues by using risk-based testing. It’s paramount to consider the impacts, ask what our priorities are, to Do the Right Thing, and not lead ourselves into a false economy when it comes to testing. I’ve seen it in many organisations over my career; it often comes from a fundamental lack of understanding around testing processes.
Quality is a mindset, and testing needs to align to the level of risk. It is often this that professional testers bring to the table, getting leadership to focus on risk, which allows us to make testing efficient and effective by focusing on the areas of highest risk, providing the data and metrics to make the right decisions about what is important, and when something is ready to be released.
This is an expensive lesson learned for the Post Office and Fujitsu with ￡1bn being set aside for financial compensation, but with customer centricity being paramount for all organisations, the devastating human impact must not be ignored.
The drive to agile delivery is absolutely the right approach as it gives everyone visibility of quality throughout, brings the customer in, and gives us the chance to evolve and pivot in short timeframes. As part of this approach, we must have professional testing involved on teams to drive for quality (but not own it) along with coaching of quality practices. I build teams to operate as a professional community with Test & Quality practices evolving constantly to deliver better and more efficient testing. Using this proven approach, we will go on to deliver ever higher quality products that delight customers and avoid expensive and reputationally damaging impacts.
Mark Winspear is a highly experienced professional in Test & Quality who has dedicated his career to building and leading teams that drive excellence and efficiency with his leadership philosophy rooted in the 'One team' principle.
Connect with Mark on LinkedIn and please do leave your thoughts and comments
Senior Test Manager at National Grid
6 个月Excellent article Mark, testing starts with Requirements Capture and Design and happens throughout the Lifecycle, both Functional and Non-Functional, ALL aspects of systems must be considered. In the case of the Post Office there were serious failings in both development and test but even more worrying was the fact that problems that were reported were being covered up, nobody must ever be afraid to "SAY IT AS IT IS" and ensure the right people are listening
Freelance Test, Data Migration and Reconciliation Specialist
7 个月Testing is still absolutely undervalued in many organisations however the catastrophe associated to Horizon is firmly due to an organisation who fail to listen and fail to deal with what could have been a run-of-the-mill issue and I say that as ex-Post Office and unfortunately I’ve seen the decision making that has pushed systems into Production despite warnings and recommendations. And it’s not a problem in doing that but when it’s done the proper support structures need to be ready and waiting with eyes open to deal with what falls out post implementation - what occurred post implementation of Horizon was startling
Senior Business Change / EPR Project Manager
7 个月Love this Mark Winspear very insightful and informative on how organisations can prevent by following the right steps to prevent