KPIs - the productivity illusion

In "KPIs - robustness, gaming and balance" I looked at how Key Performance Indicators (KPIs) can be made more robust using a "Balanced Scorecard" approach, making it difficult to improve the measure independently of achieving valuable outcomes.

There are various categories that we might use for our balanced scorecard. Flow Metrics show that work is being done and show that the backlog is being processed in an effective manner.? Quality Metrics might ensure that we are delivering work as planned without defects and rework. However, we still have a gap. These two focus on efficiency - “doing the work right”. Our Scorecard isn't balanced unless we look at “doing the right work”.

One of the first measures I am inevitably asked for in a new organization is a measure of productivity.? Any organization would like to reliably measure how much value they are getting from their development team.? Not an unreasonable request and could allow the team to build improvements that increase this value. However, once we look into this in more depth, measures of value and productivity start to become much more challenging.

Read: Metrics - friend or foe? https://www.dhirubhai.net/pulse/metrics-friend-foe-jay-alphey-chc5e/

Read: KPIs - robustness, gaming and balance. https://www.dhirubhai.net/pulse/kpis-robustness-gaming-balance-jay-alphey-wzwwf

Read: KPIs -flow metrics. https://www.dhirubhai.net/pulse/kpis-flow-metrics-wip-jay-alphey-zxnte/

Productivity measures

For Flow and Quality metrics there are a set of fairly well-established measures available and the main decision is around which to use.? For a productivity measure, the situation is less straightforward.? There is no clearly accepted metric which measures productivity.

Productivity is defined as a measure of output (value) divided by input (cost).

We can assess cost fairly reliably. We work with stable teams, bringing the work to the team. We know the cost of maintaining the team in terms of salaries and associated overheads. It seems easy to build this into a productivity measure.

The challenge however lies in measuring the value of what the team creates. Absolute measures of value are rarely simple.? Without a reliable measure of value generated, a true productivity measure is unlikely to be achievable. Even definitions of value can vary widely, before considering how to measure it.

Value: A benefit provided to a customer or stakeholder. When a product or service has been perceived or appraised to fulfill a need or desire—as defined by the customer—the product or service may be said to have value or worth. - Value Stream Consortium

Read: Agile values - customer collaboration https://agileplays.co.uk/customer-collaboration-over-contract-negotiation/

Podcast: Deliver value not just content https://agileplays.co.uk/podcast/focus-on-delivering-value/

Volume of output

Many productivity measures look at output rather than value and are therefore more efficiency measures. However, these will not balance our Flow metrics. We can make more content and make it more smoothly, but can we demonstrate that this is making more value? Measures such as lines of code are flawed and encourage the teams to create volume. Agile principles are to focus on creating value not on quantity.

Simplicity–the art of maximizing the amount of work not done–is essential. - Principles behind the Agile Manifesto

Read: Agile Manifesto: https://agileplays.co.uk/understanding-the-agile-manifesto/

Revenue based measures

Revenue-based measures seem promising. If we can assess how much revenue each Engineer generates, does this give us our value measure?

This approach might work in consulting, where an Engineer is directly generating business revenue. But in product development, an engineer doesn’t directly generate revenue. Even his/her team doesn’t. Revenue creation depends on many parts of the organisation – deployment, sales, product. This is the concept behind a value stream, where value is built incrementally across many activities.

A value stream is ... the set of actions that add value to a customer from the initial request through the customer's realization of value. - Value Stream Consortium

As a result, revenue tends to be very loosely linked to engineering activities and very strongly lagging. Although it is of course an important measure for any organisation and has high relevance it has extremely low controllability as it is very loosely linked to behaviours which can be modified.

Read: Value Streams https://agileplays.co.uk/end-to-end-optimisation-with-value-streams/

Utilisation

A widely used measure for value is to look at the utilisation of teams. Utilisation is the fraction of time that a team is busy or the fraction of capacity which is being used. We are paying the cost of the team continually, so utilisation gives a measure of how efficiently that money is used.

However, this is again a measure of output, not value. Being busy tells us little about being successful. It also does not work in an Agile development context. Backlog works as fuel to ensure that the team always have planned work ready to be executed. Therefore a well managed Agile team does not "stall" from lack of work, and so should have 100% utilisation.

Measuring waste

It seems that absolute value measures will not solve our problem. Instead, rather than measure efficiency directly, we can take a Lean approach and consider waste instead.?Remember that waste impacts the flow of value to the customer. So instead of maximising value, we can minimise waste. The two are of course not identical, but waste may give us better opportunities for measurement.

We can assess how much of the work is value, and how much is waste.? And we can do this without using absolute measures of value. We can consider the proportion of work which generates value, rather than the absolute value generated.

This is likely to be a relatively low overhead to collect the data. Since we are already tracking the work in a work management system, the challenge is to categorise the work and assess what is value and waste. This is itself a valuable exercise to undertake.

Read: Tracking Agile work: https://agileplays.co.uk/tracking-work-in-agile-development/

Read: What is waste? https://agileplays.co.uk/what-is-waste-muda-in-lean/

Strategic Focus

Engineering teams work on roadmap features.? However they also necessarily work on many other areas.? Defect fixes, customer support, technical debt reduction and so on.? We can assume that progressing the roadmap constitutes value, while other work constitute waste.?

I should be very clear here. "Waste" does not mean that we can simply stop doing the work. There is no suggestion this is unnecessary or should be removed.? However, “waste” is work which is not valued by the customer. We should therefore look to minimise this non-value-adding work.

Remember “waste” is work which is not wanted by the customer.? There is no suggestion this is unnecessary or should be removed.? It is however true that the more efficient the organization, the less “waste” work it needs to do.? ?

I generally use a metric I call “Strategic Focus”.? This represents the fraction of the work which progresses the roadmap and represents the “value” work done by the team.?This is relatively easy to communicate and understand.?

Read: Maintaining focus: https://agileplays.co.uk/in-a-volatile-world-how-do-we-maintain-focus/

I help scaling tech organisations with systems and structures to achieve repeatable delivery.

If you want to discuss how I can help your organisation, drop me a message.