On Metrics: How to Measure Agile Success?

Incremental development and improvement is at the heart of agile product development. However, embracing the agile mindset and implementing it can be two different things. How do we ensure that teams are adding more value over time? How do we ensure our new process is impactful?

?

This is where the need for quantitative and qualitative evaluation comes in. Agile metrics can help us gauge opportunities to deliver quality solutions optimally, while meeting outcomes and satisfying customer and business needs.

?

Here are some metrics we can benefit from at each level of the organization:

?

Team’s Effectiveness (in a 2 week Sprint or Iteration):

• Customer Satisfaction Score: Customers/stakeholders ranking the product increment from 1-10
• Plan vs. Actual: Predictability metric to measure amount of work planned in an iterative cycle vs. what got delivered
• Team Happiness: Team ranking their happiness from 1-10
• Flow Velocity: Average amount of work completed in an iterative cycle
• Work in Progress: Average number of stories/tasks in progress
• Cycle Time: Time spent actively working on a story/task
• Lead Time: Average time a story/task takes from Creation to Delivery (this includes wait time before cycle time)
• Code Quality: Measures ‘good’ vs ‘bad’ quality code; could include reusability, reliability, complexity etc.

?

Team of Team’s Effectiveness (in a quarter):

• Flow Predictability: How closely does a team of agile teams meets its objectives (similar to planned vs. actual)
• Flow Time: Total time from ideation to completion
• Flow Load: Total amount of work in the system at any point in time
• Flow Efficiency: Ratio of total active time (cycle time) to flow time
• Net Promoter Score: How likely are customers/stakeholders to recommend your solutions to others
• Objectives and Key Results: Evaluate key results towards achieve a set of business objectives

?

Value Stream’s Effectiveness (over a quarter, semi-annually, or annually):

• Business Agility Assessment: Measures how quickly an organization is able to respond to market changes, threats or opportunities.
• Key Performance Indicators: Measures the success of business outcomes – these depend on the type of solutions e.g. Net Promoter Score, Throughput, Tickets outstanding, Pirate ‘AARRR’ metrics
• Employee Engagement: Measures how motivated and engaged employees feel towards achieving the organization’s goals
• DORA metrics: Measures an organizations DevOps capabilities by Deployment frequency, Lead time for changes, Time to restore service and Change failure rate

?

Measuring performance is one of the most sensitive areas in every business, often subject to politics, dysfunctions and personal biases. It’s important to ask ‘What decisions will this metric help inform?’ and ‘Do we need to measure this right now?’

Goodhart’s Law states that:

When a measure becomes a target, it ceases to be a good measure.

So, it’s important to measure the right metrics, and use the data right as well. The right metrics can help us identify areas of strength and weakness, while promoting achievements, transparency and enabling conversations on how value delivery can improve.

Qualitative analysis, along with good judgment, is crucial.