Quality Perceptions

Previously, I identified two types of quality:

1. Real quality which can be found more so in industries such as manufacturing. This is where the quality is something physical, it's tangible, and you can physically hold it.

2. Perceived quality which you cant really hold but its more of a feeling. People verbalise it differently but I think there is a general theme. See if you can pick up on it...

Dan Ashby refers to it often as “Goodness” 

Rob Meaney when asked about the role of customisability in quality said “...the quality of something is the degree to which we satisfy the needs of people that have a relationship with the said thing”

Janet Gregory often refers to a cup of coffee to explain that "Quality is vague and often hard to describe"

The theme here I think is summarised best in what Chris Armstrong said “Quality is like beauty, it is in the eye of the beholder.”

The trouble with perceived quality is there are so many variations. Each customer has his/her own perception of quality in their head. These perceptions may be unknown at the time and aren't always in our full control. So how do we come to grips with perceived quality? How do we measure it? What journey have I been on? 

To understand perceived quality requires a mindset shift. This shift goes hand in hand with how Software QA has evolved. The journey began back in the days of Waterfall software development. Those of us that have been in the industry a while will remember the time where QA organisations were siloed and stuff was thrown over the wall for QA to test. First thing to hit us would be the big bulky requirements documents that we would then have to sift through and identify test cases based on the document. Next we would get the built application and ensure that it met the requirements that we had identified in the form of test cases. It was very black and white, almost tangible in a way.

Then we moved to Agile and it became about smaller bits of documentation with collaboration along the way. Now we had more freedom as QA to spread our wings. We began to find out what people really thought of the product. In some cases we realised that what was documented doesn't always capture everything we may need to look out for to assure the quality our customers are seeking.

Fast forward to present day and this collaboration is becoming more inclusive and encompassing. Today many organisations are attempting to move towards more of a DevOps approach which I believe in its purest form is about bringing development and operations together for an ever increased form of collaboration. With this in mind let me share how I am attempting to measure perceived quality.

As discussed above we need to ‘Satisfy the needs of people that have a relationship’ with in this case the software. So I was lucky in that a large majority of the software that my team works on is used by internal customers. However it is worth noting that even the external customers which use our software would interact with those same internal customers to report any issues. Therefore once I identified that what I truly care about is their (the customers) perception of quality then I figured one way to measure their perception of quality would be to collaborate with them on defects that they find. I label these ‘Escaped Defects’. This gives me a way to, at the very least, gain a pulse of what the perception of quality is from their point of view.

The next step was to bring together our understandings of the defects that are coming in. In order to do this we needed to set some guidelines on exactly how we would prioritise these defects ensuring that everyone is on the same page as to what is considered a Priority 1 (P1) and what is a Priority 5 (P5). We decided to do this using a simple impact and urgency matrix where we defined each level of impact and each level of urgency then we can easily plot on the grid where a particular defect would fall. 

This method accomplishes a few things. First, it helps educate the business on defects and how to objectively prioritise things so that not everything is coming in as a P1. Second, it gives them (the business) immediate feedback as they log issues which in turn should influence their perception of quality right out of the gate. What do I mean by this? How does logging a defect and assigning it a priority based on some criteria give the business immediate feedback, influencing their perception of quality? Well, If the people logging the defects are seeing that a majority of the defects being logged are P4’s and P5’s rather then a ton of P1’s and P2’s then subconsciously it would seem that the perceived quality of the software would go up. Lastly, prioritising the defects allows us on the product team to address the defects that would improve the perceived quality in the customers mind first. This simply means addressing the P1's and P2's before the rest. Thus shortening the time it takes to see the perceived quality of the application go up.

The final step is to demonstrate progress on issues being resolved. For this I give key stakeholders an update on issues that are currently being worked on at every triage meeting and inform them of progress. They also can see clearly the progress of any issue they have logged in the past and see where it is in the process. As the weeks and months go by we have seen the P1s shrink and we have even managed to get our P2s down to a number that can be counted on one hand. Thus the hope is in the next few weeks we can begin to address some of the P3s. Now Im not saying that more P1's and P2's wont come in, they probably will, but at least we show progress in the right direction and its very visible and transparent hence its highly likely that through this our customers expectations are more likely to be met.

Thus it is my hope that while we live in a software world of perceived quality, through collaboration and working together with the operational side of the business we can find creative ways to bring about that ‘Goodness’ that Dan Ashby talks about, We can ‘satisfy the needs of the people’ as Rob Meaney described, and we can at least work towards making the ‘vague and hard to describe quality’ that Janet Gregory talks about a little bit less vague and a bit easier to describe. Because at the end of the day quality to me is creating harmony, that works!

----

Authored by: @SufyanFarooqi

For more on this topic check out the round table discussion below that I had the privilege of participating in with this esteemed panel: Steve Watson, Marie Drake, Oana Rusu, Seema Prabhu, Rafaela Azevedo, & Nicola L. Martin. Hosted by Quality Advocates, Billy Senior :