DORA Report - 2024

DevOps Research and Assessment?(DORA) 2024 report on "Accelerate State of DevOps" is out. Please refer to the link in the first comment to get your copy. This is just my personal commentary on this year's report. It does not represent my employer's views.?

I have closely followed the DORA metrics published by the Google Cloud team for years. They have the most extensive data collection on this front. This is more than just a report of data collected, as you might have seen with StackOverflow yearly reports. The Google Cloud team adds much color(their interpretation) to the data. As they say in the report, for any data to be meaningful, the context matters. I agree with that, but I suggest we also look at the data from our own lens. The business demands might often require different metrics than what DORA calls "Elite" performers.?

Core Metrics:

DORA has many metrics. But the following four are the key ones:

• Deployment Frequency?
• Lead Time for Changes
• Time To Restore Service
• Change Failure Rate

In most of the earlier reports, these metrics moved in the same direction. As in, whatever DORA defines as Elite has the best numbers in all four categories, and then it progressively goes down from Elite to High to Medium to Low. Low was always the worst performer on all four. This is a lot easier to understand. But this year was a surprise.?

Whatever DORA calls medium performers because of their speed of delivery had almost the best change failure rate and a decent recovery time. This is where the data is complicated to explain. The report attempts to explain it with the reasons they think are valid. However, we can only explain it by talking to the respondents. As a practitioner, I can say that most of us have become better at "Recovery time". Downtime is not accepted today, irrespective of whether you deliver every minute or every month. Business users demand high availability of the applications. From the days of "Rollbacks are not documented" we have moved to "Rollbacks are well documented and mostly automated". ?

DORA reports are used to publish the percentage of respondents from each category. They have avoided it. It is probably not an oversight. There is perhaps some view on that data that is not fully explainable. To be clear, I am not implying any harmful intentions on the part of the team. It is just intriguing that that data is not included.?

AI:

As with everything this year, the report spends so many pages on AI. AI is redefining how every developer looks at their job. As product managers, we are all asked every day internally within our companies and by our customers, "What is new with your product? Anything on AI?". AI will improve all our jobs, make us more productive, and have a net positive impact on all of us. Some of the popular tasks where AI was used (not many surprises here):

Interestingly, AI was more helpful in the work that developers thought was valuable, probably "code generation/code explanation/ documentation, "and less so valuable in toilsome work like meetings and bureaucracy.

There is so much more on AI in the report. I will let you feed it to an AI and get your tl;dr :)

Platform Engineering:

There is still an emerging idea in large enterprises. Every organization will move towards platform engineering at some point, irrespective of the development organization's size. The self-service method is best suited, with individual teams utilizing the platform for most tasks with a few exceptions explicitly added for their team.?

The report notes that organizations with platform engineering teams sometimes reduce the throughput. The reason cited is that org-wide platforms mandate many checks and balances, which are also critical from an audit and compliance perspective.?We probably have to find some balance here.

Leadership:

There are additional observations on how a user-centric approach to internal and external products is critical and how to lead in tough times. I am skipping my commentary on those as they seem to move toward leaders in the org, whom I hope will read the full report and will have better advisors than me to say more about it :)?

Conclusion:?

If you are part of platform engineering, Developer Experience, DevOps teams, or someone who creates DevOps products, I suggest you read the full report. It is a 120-page report, but the core content is in the first 60 pages. I also recommend you consider your end users' experience with this data to understand what will work for you.?

For example, ask me about Mainframe application development. In that case, mainframe teams might want to reach the metrics of High/Medium with more effort toward maintaining the system's stability compared to the delivery speed. That does not mean I suggest staying at the "low" levels. Every team has to find its own balance, and it just does not make sense to run behind the metrics and lose focus on the business needs.?

If you have read the report, please share your thoughts in the comments.