Metrics for High Performing IT Teams

DORA Metrics

How do you measure the productivity of developers? Google has a clear definition of how to measure ?the success of a DevOps team. It uses 4 key metrics called DevOps Research and Assessment) DORA metrics?that lays the foundation of an effective DevOps Team. To appreciate these metrics, keep in mind the definition of DevOps, used by Google: “an organizational and cultural movement that aims to increase software delivery velocity, improve service reliability, and build shared ownership among software stakeholders.” This definition further goes into how DevOps teams should work to “improve the speed, stability, availability, and security of your software delivery capability.”

These metrics span 3 teams in a ny organization involved in software development: software development, software deployment, and service operations. The four key DevOps DORA metrics are:

• Lead time for changes
• Deployment frequency
• Time to restore service
• Change failure rate

High Performing IT Team

The key point is that every metric is and cannot be completely owned by the one of the 3 teams. A successful DevOps team understands the shared ownership of these objectives. But what do the high performing teams do?

• They release many times per day.
• The lead time for changes and moving into production is less than a day.
• The time to restore service is less than an hour (low performers take between a week or a month).
• The change failure rate is zero to 15%.

These elite performers reach corporate goals because they do well at the following metrics:

• Profitability
• Productivity
• Market share
• Number of customers
• Quality of products or services
• Operating efficiency
• Customer satisfaction
• Quantity of products or services provided
• Achieving organizational and mission goals

All of these pour into the belief that software delivery and operations (SDO) performance predicts the whole organizational performance. There is a further correlation between your SDO’s performance and your DNA of cultural performance. Your culture should be mission oriented and not control oriented.

All these elite performers share in fostering a climate for learning, highly participatory retrospectives, and the encouragement of trust, voice, and autonomy. But it’s also not just about the people. Tech is just as important as culture in DevOps.

How to foster a high-performing culture?

Elite teams understand who does what and automates as much as possible. Out of the 4, the toughest metric is the lead time. Related questions to this metric are: How long would it take your organization to deploy a change that involves just one single line of code? Can you do this on a repeatable, reliable basis? The answer lies in focusing on the following aspects.

1. Garbage collection:?Who should I be billing for this virtual load balancer or database instance? What would happen if I deleted this service? Are people still using it? The platform should ensure that every virtual resource is assigned to either an app or the platform itself.
2. Making changes:?If an app has a vulnerability, how do I fix and deploy it? If I need to update this dependent service, where is the source code? It should be possible to redeploy any app at the click of a button.
3. Multitenancy:?How do we enable developers with self-service deployments or configuration? Making AWS and Kubernetes multi tenant is difficult but it’s an essential requirement of any enterprise platform-as-a-service (PaaS).
4. Managing complexity:?Making sure the stack is up-to-date. How will we hire people in 15 years that know how to work this? For example, all apps must be built on predefined approved runtime stacks that PaaS operators can patch and redeploy on demand.

The Architecture Behind Elite DevOps Teams

There are architectural outcomes that allow teams the flexibility needed for high-performing DevOps without security risks. The key questions to ask here are:

1. Can my team make large-scale changes to the design of its systems without the permission of somebody outside the team or depending on other teams?
2. Can my team complete its work without needing fine-grained communication and coordination with people outside the team?
3. Can my team deploy and release its product or service on demand, independently of other services the product or service depends on?
4. Can my team do most of its testing on demand, without require an integrated test environment?
5. Can my team perform deployments during normal business hours with negligible downtime?

Team Happiness: DevEX

It is not the individuals but the team and organizational culture that make or break these elite DevOps performers. The culture of "Psychological Safety" where team members feel safe to take risks and be vulnerable around each other — have a greater capacity for dependability, structure, clarity, meaning and impact on the organization. The other benefits of such a fostering culture include reduction in burnout, deployment pain and, rework.

Today's concept of metric therefore moves to DevEx - Developer experience, as it centers on human beings, is inherently sociotechnical. Still, the focus is skewed on the technical side. The new concept must take into account a few important factors:

• Software development is more complex today
• Remote and hybrid work are prevalent

DevEX focuses on the lived experience of developers and the points of friction they encounter in their everyday work.” It’s not just about productivity, but increased efficiency, product quality and employee retention.?DevEx has also been defined?as encompassing how developers feel about, think about and value their work — not exactly easily measurable subjects, which may be why, unfortunately, most companies aren’t looking to measure them. You can visualize them as consisting of 3 important components:

• Feedback loops?– waiting time for developers to get done their work and how streamlined teams can shorten that time
• Cognitive load?– in the ever-growing complexity of the cloud native world, organizations should look to limit hurdles to delivering value to customers
• Flow state?– when developers “get in the zone,” with limited distractions — meetings, unplanned work, ad-hoc requests for help — they feel energized by a greater sense of purpose

Individual or Team?

Developer experience is highly personal and contextually dependent. The framework is unique in focusing heavily on the individual. But that creates a challenge around how to measure the individual but work to improve the collective experience. Usage of surveys as an important “fast and accurate” measurement tool. After these carefully designed surveys — asking things like “Based on X, how likely are you to…” — are regularly run, break down results and Net Promoter Scores (NPS) by team and developer persona, advises the paper. Noda clarified in our interview that these surveys should be anonymized or aggregated. It remains unclear how anonymous it can be on the average “two-pizza team” of five to nine people, and if it really can be individually actionable to aggregate results.

A key measurement of developer experience is how good you perceive you are at your job — because feeling good at your job is highly motivational and signals both a lessened cognitive load and an optimized flow state. However this measurement brings its own slew of?implicit biases?that increase with?intersections across demographic, role and experience.

Focus on Flow

Embracing your flow doesn’t just mean going fast — it’s as much about reducing friction and frustration for?more focused and happy developers. Flow metrics is about finding a sustainable pace that allows you to keep going infinitely.?It’s not necessarily about speeding up. Yes, they can help you increase your velocity, but it’s about finding a pace that works for you. making lead time also an individual metric.

• Reach the required level of consistency before aiming for high pace that is sustainable in your organization. A controlled growth that is sustainable is better. A big bang release is not as good as 6 small releases.
• Progressive delivery?is a series of technological solutions to help decrease cognitive load, allowing teams to roll back changes more easily.?
• Observability?and monitoring are also essential so bugs and causes of outages can be uncovered much faster.

Flow state helps to utilize developers' time well and contribute to their well-being. Context switching wastes resources.

To sum up:

• Build the foundation of DORA metrics
• Shift focus to individual developer productivity to team
• Give a sense of belonging and achievement to the team
• Focus on the experience at 3 levels: individual, team and organization wide