KPIs - Flow metrics and WiP
In "KPIs - robustness, gaming and balance" I looked at how the use of balanced KPIs could help achieve robustness through an "Engineering Scorecard". I proposed three categories of KPIs we could use (not suggesting this is an exclusive set) - Flow, Quality and Value.
Now I will dive in on one category - Flow metrics - and look at some ideas of KPIs we might use. Remember that we are aiming for four characteristics of good metrics:
Read Part 1 - metrics, friend or foe: https://www.dhirubhai.net/pulse/metrics-friend-foe-jay-alphey-chc5e
Read Part 2 - KPIs: robustness, gaming and balance: https://www.dhirubhai.net/pulse/kpis-robustness-gaming-balance-jay-alphey-wzwwf
Read: Engineering Scorecard: https://agileplays.co.uk/an-engineering-scorecard-as-common-language/
Flow metrics
We are looking to achieve a continuous process of delivering value to the business at a sustainable pace.? Lean uses the term “flow” for the movement of value from the creator to the customer.? The analogy of flowing water describes the approach very well. Water, in Japanese philosophy, represents adaptation and progression.? Water adapts to obstacles and flows round them.? Rivers move forwards to the sea.
Water flow can be smooth, or it can be turbulent.? Our objective is to remove barriers where possible.? This encourages value creation to be as rapid and smooth as possible.? Once we have identified how value flows to the customer, we can identify barriers.? These disrupt or slow the smooth flow.? We can then address these barriers.
We are reducing the time line by reducing the non-value adding wastes Taiichi Ohno
Read: Value Streams: https://agileplays.co.uk/end-to-end-optimisation-with-value-streams/
Read: Promoting Flow: https://agileplays.co.uk/seven-lean-principles-to-promote-flow/
Cycle time / throughput
Probably the most popular metric adopted to get an overview of how work is flowing through the business is cycle time. This is the average amount of time to complete a work item. If you imagine you are standing at the end of a conveyor belt, counting how long between items coming off the belt, that is cycle time.
Cycle time is also represented as throughput, the number of items delivered per unit time. Throughput is the reciprocal of cycle time. If we produce 5 items a week (throughput) then it takes on average one day to produce one item (cycle time). Personally I prefer throughput as it avoids confusion with other time measures such as "lead time" (which I cover below).
Like many Lean measures cycle time comes from a manufacturing environment and is an easy measure on a conveyor belt. It is harder to use in a software environment. Remember that our objective with we want a clear measure which is understandable by the whole organisation as part of the “language”. In manufacturing, all items are equal. If we’ve rightsized the work, we may indeed have very similar sized items and be able to count them. This simplifies many areas, notably estimation and planning, possibly leading to #noestimates. But in most cases, work items are different sizes and larger ones will, naturally enough, take longer.
Cycle time is highly dependent on what we are measuring. The throughput of lines of code will be higher than backlog items, which will again be higher than features. But the variability in size and the amount of data will be the opposite - many lines of code to measure, few features of very varying size.
This makes cycle time hard to use as an organisational language and a KPI. The higher level elements about which the organisation cares are likely to have varying size and infrequent enough to be affected by statistical variation.
Cycle time (which is a key DORA metric) is a more valuable internal as a "fast feedback" measure to assess team flow at the backlog item level than a communication tool.
This tends to be weak in measurability and relevance.
Read: Rightsizing: https://agileplays.co.uk/what-do-we-mean-by-rightsizing/
Read: DORA metrics: https://agileplays.co.uk/continuous-improvement-using-dora-metrics/
Velocity
We have seen the limitation of cycle time measuring unequal sized work items. One solution would be to take the item size into account. Rather than counting the work items completed, we could sum up the sizes of the work items.?
This leads us to looking at Velocity as a measure. Velocity is the amount of estimated work we complete divided by the elapsed time to complete that work.? At first glance this seems a sensible enhancement.
However, the weakness here is that there is no absolute measure of size of work.? We would count the amount of work delivered based on an arbitrary relative sizing scale. Velocity might increase or decrease, not because the team delivers more or less, but because teams use different values and the units change and evolve over time.
For a KPI this really falls down on measurability and robustness.
Read: Velocity as a tool not a weapon: https://agileplays.co.uk/effective-use-of-velocity-for-planning/
Lead time
Lead time is often confused with cycle time, but is a distinct measure. Remember that cycle time is the gap between items coming off the end of the conveyor.? Lead time, by contrast, is the time from an item being put on the start of the conveyor to it coming off the end.?
The biggest challenge here is agreeing what is the conveyor.
Defining lead time in a way to give a valuable KPI is a real challenge. Change and reprioritisation can make a huge difference, as can batching releases. Do we measure completing work in a day or release monthly?
As with Cycle time, Lead time can be very effective as a fast feedback measure. DORA looks at a very specific measure of the time from work being committed to getting into production.
In general though, measurability is the challenge here.
Work in Progress
My personal preference for a KPI is to focus on Work in Progress. This seems a rather less popular measure, possibly because of its background in Lean rather than Agile.
Work in Progress is the amount of work which has been started but not yet delivered. I have always found it relatively easy to communicate. It represents cost which has been spent, paying development teams to create code. However, there has been no benefit gained yet since the code is not in the hands of customers.
In general, work in progress is a good choice of measure. It is relevant by representing unrealised cost, while also being controllable by being strongly affected by good flow techniques. In particular, it encourages small batch sizing, frequent releases and minimising parallel working.
It is also easy to communicate. The main challenge around measurability is to ensure a really clear definition. Many people meet work in progress when discussing the number of Stories open during a Sprint. That's not a very useful measure for a KPI. In general we want to focus at a higher level. This will give a measure something like this:
The total amount of work done on incomplete features
Read: Work in Progress https://agileplays.co.uk/why-work-in-progress-is-key/
Read: WiP and DORA: https://agileplays.co.uk/is-dora-missing-a-wip-measure/
I help scaling tech organisations with systems and structures to achieve repeatable delivery.
If you want to discuss how I can help your organisation, drop me a message.