Monitor System

Questions of the week:??How can we effectively monitor the progress of a system?

I write this article based on the insight of my conversation with my team in our monthly 1-on-1 call. I speak with each one of my team members. This article is to share the question of this week and our method to work on it. You can subscribe to the Building Our Team newsletter (2450+ subscribers)

Photo by AI (Microsoft Bing)

How can we effectively monitor the progress of a system?

Measuring the success of a system early on can be a big challenge and confusing at times.? ?

My team can watch the detailed answers in the culture call of December 2023

I want to share some insights from the answer, which is important to understand if you want to ensure your system succeeds. A bit of background: this question was asked by a QA Lead. So, her team wants to ensure zero or close to zero bugs in our software.

First, you must start by applying 80/20 rules to pick the metric. I don't measure one dozen metrics. I recommend choosing only one metric that has the maximum impact. At a maximum of 2 to 3 metrics in case you cannot boil it down to one.? ?

The key here is that there are two metrics to look for: lead metrics and lag metrics.

Lead metrics are early indicators or predictors of future success. They are measured before the actual outcome is achieved. Lag metrics, on the other hand, are measured after the fact, once the outcome has occurred.

Lead metrics are often actions you have under your control, and you can do it without any challenge. Lag metric mostly has external factors, such as other teams and business conditions.?

For example, If you are on the journey of getting fit. A lead metric will be going to the gym 4 times a week, Not eating after 7 pm, drinking 8 glasses of water per day, etc. The lag metric will measure your weight, inches, and blood work every month. If you only measure the lag metric, you may not be successful.?

Once you reach your ideal weight, you keep measuring your lead metric because people will tell you that you are looking fit, and you have to measure your lag metrics monthly.?

Thus, for our QA lead, the metric to measure could be as follows and more focus on the lead metric because the lag metric is a by-product of the work you do.

Lead Metric?

Test Coverage:?The percentage of the software's codebase that has been tested.

Test Automation Coverage:?The percentage of test cases that are automated.

Code Review Findings:?The number and severity of issues identified during code reviews.

Lag Metric

Defect Count:?Number of bugs per sprint

Defect Density:?The number of defects (bugs) per unit of code or functionality

Bug Escape Rate:?The percentage of bugs reported by customers or found in production after the software release.

For some of the metrics above, I used the help of ChatGPT. You can also do the same.?

If you liked this, why not share it?