Quality Attributes: Judging goods with tolerance

Before starting the reading check this other article Quality Attributes: What are they? Should they be that hard to elucidate? in case you have doubts about Quality Attributes.

And one more thing: Thank you so much @Carlos Amselem for motivating me with the subject of this article and shedding light on my knowledge toolbox.

It is time..

During school time (depending on where you studied) you had a scale with a tolerance range with the score needed to be approved or to fail. Right?

Any score between 70% and 100% would be acceptable to succeed. Not all the students got 90%, 100%.. everyone who reached at least a 70% score passed and it was ok, acceptable.

Now imagine a student that does not care much about getting a 90% score and reaching a 80% score is a good target (from their point of view) because in the end the student still passes and does not put too much effort into it. The student believes that to reach the score of 90% it would cost a lot and the benefits would not be worth it. 

Going a little bit further we see that the student still passes by getting a 70%. They would definitely not be the best of the class but the score would still be accepted by their parents without many complaints. In the end the student sets the target based on the judgment of their parents: how effective the student is.

The parents are like the stakeholders, they invest so they judge what is good and in this case the quality that the parents look for is the Effectiveness of the student. 

Check the scale:

Definition with the meaning of each percentage value in the scale:

Failure (< 70% ): If the student does not get to at least that value, then they fail.

Target (~ 80% ): It is where the student plans to get.

Stretch (> 90%): Crossing that point only in case there is no additional cost. The benefits do not make up for the cost it implies (from the student's point of view).

Note 1: The student sometimes cannot reach the max score in all the classes but in some they could and this is a clear trade-off because sometimes the student does not have much time to study everything then the effort is put into the most important tasks. 

Remember: it is hard to maximize them all, like characteristics in softwares. 

Note 2: The scale sets Failure, Target and Stretch, but any other definition would be acceptable, such as: OK, DESIRABLE, ACCEPTABLE and NOT OK or FAIL, REQUIREMENT AND PERFECTION. The idea behind the scenes is to make you, the reader, not to think only in absolute numbers to reach some response measure. The software can succeed in a scale of tolerance range and still save some money.

The subject we will talk about works exactly the same way: succeeding in Software Delivery by keeping our response measures inside a scale with a tolerance range. The proposed scale could help us to better handle the cost and schedule.

The advantage of following this approach is that you can achieve the quality goals in steps. You play with the range tolerance in the scale and set expected values to each iteration and incrementally you reach the target of the system in the tolerance range. Sprint by sprint the agile development team could benefit from it. You are not only giving more freedom to the development team but also if you are in a leading position you can decide when to stop the improvement (cost vs schedule). After all, if you get close to the stretch and it starts getting expensive and this adds little value to your application, you should stop that effort and instead use the team’s time and energy to work on something else entirely. Architects in the end very likely should not maximize all the quality attributes, the pain is not worth it.

Make clear you understand the student's effectiveness and their parent's expectation in the introduction because Quality Attributes can have response measures just like that. 

Now it is time to dive into the purpose of the article.

Suppose we are about to refactor an existing Cinema Ticket System in order to make it competitive again (it was a pioneer in the past) or it will be out of the market soon. How could we go to the next level? What are our stakeholders expecting from the system which they are paying for? 

The goal of the system will still be the same, to manage cinema sessions by providing the users functionalities such as purchasing tickets, reimbursement, changing movie sessions, etc. Bearing in mind that in the end our success is based on how the users will judge our system, like any other good they have acquired during their whole life and in order to reach their expectations we have to understand the users and how they plan to use the system by grasping what they value most through the use of Quality Scenarios. Here we have some scenarios based on how the User Stakeholders may pay attention to when they think about judging how good the Cinema Ticket System is:

• When I purchase a ticket, I pay attention to how many steps I take to get my ticket purchased; QS-1
• When I purchase a ticket, I pay attention to how many different options I have in order to pay for the tickets (e.g. credit card, PayPal, Skrill, Stripe, etc); QS-2
• When I purchase a ticket, I pay attention to how fast the website responds to my requests; QS-3
• When I access the ticket platform, I pay attention to how safe I feel when sending sensitive data through the network; QS-4

Checking the given cases we could highlight five Quality Attributes: Usability, Extensibility with Interoperability, Performance (response time) and Security. These five cases provide five different characteristics that will be judged by the stakeholders are in the lead category, that contains the characteristics that the system should be known for. In order to ensure the five expected Quality Attributes in the lead category some design decisions have to be taken and the Quality Scenarios will very likely to become Architecturally Significant Requirements.

We could write many more cases and therefore many more Quality Attributes, but the article is purely theoretical. The other Quality Attributes are up to you to post them in comments, then we could discuss further. 

Note 3: Extensibility and Interoperability act together in order to extend the payment options by adding adapters to connect the application to third party payment transaction services.

Note 4: Bear in mind that everything in architecture comes at a price and with each choice something must be renounced, also known as trade-offs. You cannot simply maximize all the Quality Attributes because likely one characteristic will overlap the other.

Check the table below where it shows exactly that each choice made (likely) means renouncing to another. 

Usability vs Security trade-offs

• In order to give users a better experience the system will now leverage a third party authorization service, delegating its responsibility to other systems such as Facebook, Google, LinkedIn, etc.

Pros 

• Effectiveness: Fewer (hindrance) steps to get registered/logged into the system;

Observation: In the worst case the user will need to add some credit card or configure a third party payment transaction service (PayPal, Skrill, Stripe, etc);

• Learnability: Easy to use after using another existing account in a different system;

Observation: More security steps should not be used because of the difficulties (or bureaucracy?) they present users. E.g. Multi-factor authentication;

Cons 

• The system is only as secure as the weakest link (third party);

Observation: It is not new that a few years ago Facebook and LinkedIn had their services hacked and it allowed an ill-intended person to use accounts on behalf of others;

Interoperability vs Performance trade-offs

• In order to make the users feel more secure and comfortable, the system offers the choice of paying for the tickets by using third party platforms, such as PayPal, Skrill and Stripe;

Pros

• The user can delegate sensitive information to stricter and more reliable systems;

Cons

• The system is only as fast as the slowest point (third party);

Observation 1: To confirm a successful purchase the Cinema Ticket System has to receive a positive feedback from the third party system and this can take a while;

Observation 2: Depending on the integration protocol, this communication can be easy or difficult (e.g. REST vs SOAP)

We could spend an entire day writing about trade-offs among quality attributes: Security vs Performance, Reliability vs Cost, Interoperability vs Cost, Interoperability vs Reliability. You, as the reader, maybe have been thinking that we have not talked about Reliability (including Availability - a lead characteristic of any system like this in this market) and that is true, but once again, it is purely theoretical. We count on you to discuss further. 

In the end, Quality Attributes trade-offs are based on the characteristics that matter the most, based on the user’s judgment of how good that product is.

Now let's understand the tolerance range by choosing some of the Quality Attributes identified by the Quality Scenarios with the User Stakeholders by providing an acceptable scale to succeed in the Cinema Ticket System project.

These days the Cinema Ticket System is way behind the competitors and the feedback received is really clear: day after day users stop accessing the platform. This decline is the main reason for such investment in refactoring the system. If the system cannot respond quickly to the market changes, it is literally out of the market.

Let’s understand how much we are lagging behind our main competitor and where we want to get. Sometimes getting there through iterations is acceptable. 

Usability QS-1

• Gist: Reduce the number of steps to buy tickets;
• Current: 6 steps (log in, find the movie session, select the session, go to the cart, set the payment method, make the payment);
• Competitor: 5 steps (log in, find the movie session, select the session, go to checkout, visualize cart and pay); 
• Target: 4 steps (find the movie session, select the session, go to checkout, visualize cart and pay) - in this case the user has already been authorized by some social network to operate on behalf of the Resource Owner;
• Stretch: 3 steps (select the session - most popular movies on homepage, go to checkout, visualize cart and pay) - in this case the user has already been authorized by some social network to operate on behalf of the Resource Owner;
• Failure: Greater (>) 5 steps (The target but with cart visualization and payment apart from each other + the user has to log in);

Performance QS-3

• Gist: Reduce response time right after paying for the purchased tickets;
• Current: 5~10 seconds (waiting for third party systems);
• Competitor: 3 seconds (it seems to be async but the user waits for feedback on the screen);
• Target: GT (>) 1 second LE (<=) 2 second (some CPU cycle or any another resource usage could increase up to 2 seconds); 
• Stretch: ~ 1 second (right after payment you already have the ticket, only in case the payment fails you receive a SMS telling you that it was not processed);
• Failure: Greater (>) 3 seconds (we cannot follow the competitors, we have to beat them);

Extensibility QS-2 (adapter to interoperate with Upstream services as a Conformist application)

• Gist: Add new adapter to connect the application to some new payment transaction service provider introducing new defects into the system ;
• Current: ? (the application is not extensible);
• Target: 0 new defects introduced;
• Stretch: 0 new defects introduced;
• Failure: Greater (>) 1 new defect introduced;

Have you realized how good it is to have a scale of tolerance? You could even have an incremental target. Suppose you want to increase the throughput of the purchasing ticket functionality of your application but you do not have enough resources (either money or people); then you set three checkpoints, one for each of the next iteration. 

In each of these iterations you can increment a new tactic (design decision) in order to reach the expected 40 requests per second right at the end of the third iteration.

This strategy based on Iterations is flexible and helps you buy some time in order to reach some kind of “impossible” goal. This way, by spreading out the targets through three iterations you can better manage the resources and the schedule.

Now you tell me: how many times have you dealt with some absolute response measure and failed to reach it in the expected deadline? Or have you ever tried to maximize all of them instead of using a tolerance range to get them all without compromising the delivery?

Write in the comments how the scale with iterations and tolerance range could help you deliver products that are really good.