Optimizing Bugs Fix Policy

I am sure you are familiar with the following scenario: a user is reporting to your Support team that something is not working for him as expected. Your Support team investigates the issue and agrees that there is a bug in the system. They open a JIRA bug to the R&D department with all the information they have collected, as expected from them. But then… a furious argument begins on the ticket.

Support is saying that they think R&D should solve this bug within 7 days. The Customer Success Manager is saying this is a critical customer just before renewal, and therefore we need to make all of the efforts we can to solve it within 48 hours, but R&D doesn't see this as an urgent matter and thinks the bug should be solved within 30 days.

Who is right?!

Well, everyone :) 

Each stakeholder is looking at the bug from their perspective, which makes everyone right (but also makes them all wrong simultaneously). 

So what do we do? We look in this article and learn how to optimize our company's fix policy and decide on the most accurate one for every single bug.

What is our motivation for doing it?

• A common language between relevant stakeholders
• Single Source of Truth
• Improve ticket prioritization
• Removes the guesswork and make things more convenient
• Standardize SLAs & data
• Help us monitor the standards we set
• Reporting and analytics

How do we start?

The most crucial thing is to sit with all the different stakeholders (Product, CS, Sales, Dev) and compile a list of important aspects of your company's work, this could be (but not limited to): Account importance, ACV (Annual Contract Value), Renewal date, How many accounts/users this issue is impacting, which component in our product is affected, does a workaround exist, was this bug re-opened in the past, and more.

Then, you need to decide on scoring buckets for each aspect we chose as important for our company. For example, accounts could get an "importance score" between 1–5 points, while the boolean "bug re-opened" field will get a fixed score (for example, 30 points) based on the answer (Yes/No). The end goal is to have a table that will semi-prioritize your most important aspects.

3. With the help of the stakeholders, decide how to calculate the relevant aspects that are important for you to calculate the overall score. 

For example, as you can see in the image above, the CS and Sales stakeholders have decided that Microsoft was given the Priority Bucket A with a score of 75, Tesla was given the Priority Bucket B with a score of 50, and Facebook was given the Priority Bucket C with a score of 25. The decision might be because the account is in Hyper care or before renewal, or simply because their ACV is high and their logo is important.

The same way Product has decided that Feature A will get a Score of 1, Feature B will get a score of 4, and Feature C will get 2. This could also be because of the importance of the feature, how crucial it is for the users' work, and more.

4. The next step is actually to build the formula. We need to define a scoring algorithm based on the fields we decided are important for us. The calculation, ideally, should be the multiplication of your ranges (the 1–5) and then adding the fixed score. Your overall score should define your desired fix policy. 

We can say that a 48-hour bug score will be between 101–330 (330 is the max that can be based on the example mentioned above), a 7-day bug score will be between 56–100, and a 30-day bug score will be between 0–55 (0 is the min score that can be based on the example mentioned above)

For example, if we look at the example from the screenshot above, we can say that the formula will be: (Component * Account) + Bug Re-Opened + Workaround Exists. Let's take a look at a few use cases:

a. Feature C, Account Priority A, Bug not re-opened, Workaround exists: (2 * 75) + 0–25 = 125 -> 48-hour Fix Policy

b. Feature B, Account Priority C, Bug not re-opened, Workaround doesn't exist: (4 * 25) +0 + 0= 100 -> 7-day Fix Policy

c. Feature A, Account Priority B, Bug re-opened, Workaround exists: (1 * 50) +30–25 = 55 -> 30-day Fix Policy

• For further information about Incident Management, Automation, and Learning abilities, read here:

5. You might ask yourself how do I verify my scoring brackets make sense and are correct? Well, that's a great question!

We take the last few dozen bugs that were opened in the last couple of weeks and start running the formula on them. According to the results, we adjust the formula, the value of each part in the formula, and the overall scoring brackets until we feel that the algorithm makes sense and answers the vast majority of bugs (I would recommend more than 85–88%).

Let's start automating stuff!

Everything was nice and simple until now, but manual. Very very manual. Why not make the formula run automatically based on the fields that the Support team enters into JIRA when they open the bug?

1. We begin with adding the relevant fields to JIRA, based on the relevant aspects that we mentioned as part of the formula.

2. Using webhooks and REST API requests between JIRA and our internal codebase, we can create the formula to run automatically, calculate the value and according to the bracket - update the Fix Policy and the Potential Fix Policy Date (which is today + the fix policy stated).

3. An automatic update is being sent as a comment to JIRA, with the relevant information about the scoring formula:

Or with a message in case someone entered a manual policy that is different from the algorithm (and the user needs to add a special label to mark it as such):

Maintenance and Reporting

We can't just sit around and wait, as our product is evolving and changing, so we must update our formula on a quarterly basis with the relevant stakeholders. Every time we update the formula need to make sure that the algorithm still makes sense and based on examples we take.

In addition to that, we can export the information from JIRA, and with the help of the label mentioned above, we can work on constantly improving our algorithm and make sure it is always up to date.

Optimizing your bugs fix policy is a critical path in the success of every company, will solve a lot of pain points with multiple stakeholders in your organization, and improve the amount of time it takes you to fix urgent bugs in production.