Metrics for Managing DevOps Implementation

Today’s Business Environment is forcing companies to shift their focus from stability and efficiency to agility and innovation. This is forcing companies to change the way they operate. In this competitive environment, where disruption in IT service is forcing CEO’s to apologize to their Customers, IT has to change. They can no longer operate in the Dev and Ops silos. One proven movement that makes IT receptive to change is the DevOps movement.

DevOps basic principles are

1. Systems way of Thinking
2. Feedback
3. Culture of Trust and Experimentation

But how do you measure your DevOps implementation.  Which metrics should be tracked which gives the organization a measure to evaluate their DevOps Implementation?  Per my view below are the 3 simple metrics that should be tracked. These metrics are in line with the DevOps principles.

1. Lead Time: This is the time it takes from Code Commit to production Deployment.
2. Mean Time to Repair (MTTR): This is the average time needed to repair a problem. This time should be measured from the time the problem occurred and not when it got reported.
3. Frequency of Deployments: Number of deployments being done in the defined time-period.

The Lead time metric will tell you how efficient your Build, Test, Release and Deployment process is. If this metric is reducing over a period of time then you are on the right track.

The MTTR will tell you how efficient your Ops team is in detecting the problem, reporting the problem (feedback loop) and the efficiency of your Dev team in fixing the problem. 

Frequency of Deployments will tell you the overall efficiency of your DevOps Process. An increasing number of deployments over a period of time means you are making continuous improvement in your DevOps process.

Other metrics such as Failure rate, Change Volume, Availability do not add any value. However these metrics (if collected) point out the fact that the Org is more interested in stability and efficiency rather than agility and innovation. For e.g. a declining Failure Rate points out that you are not experimenting   and innovating (experimentation will lead to Failures).

Metrics should also follow these basic principles else they become work items on their own requiring significant amount of effort to collect them.

1. They should be easy to collect.
2. Collect no more than 3 metrics for the domain you want to measure.

The above Metrics are what we call Technology Metrics, but these Metrics will not help your Business to make decisions. Your DevOps initiative has to add business value. Any IT initiative that does not add Business Value makes no sense. So what are the Business Metrics that you should collect for a DevOps Implementation?  Well “It Depends”, it depends on the Business you are in. Business Metrics for an e-commerce company will be different from a government organization implementing DevOps. Well the U.S. Customs and Border Protection are pretty active in the DevOps space.

So go ahead and develop your own Business metrics as per your need. Till then Happy DevOpsing.