Questions and Answers on the Metrics You Use

Last week I conducted a workshop on everything a scrum team can measure and my oh my it was an event! Why, Anna, why? Because I was talking to the data team. So literally everything they do is data driven, data sponsored, data resulted.

What portion of your team's time is dedicated to actively progressing 'in-progress' tasks? Is it 70%, 75%, or 80%? Some believe it might be as minimal as 2%, with 40% marking the upper limits. This phenomenon describes the concept of flow efficiency. So let's start the article with this particular metric and how two frameworks use it.

Scrum Example:

In Scrum, the primary focus is on Sprint completion and its efficiency. The team examines how many value bringing items were completed in the given time-frame.

(in other words, you can be very busy working on something, but are you working on what brings value?)

Flow efficiency in Scrum would look at the flow of sprint backlog items within the Sprint. Please note, that a way to look at this metric in scrum is number of value bringing items compared to lead time, but here we can start talking about what actually brings value (well, a reminder that every user story has to be INVEST where one of the letters actually talks about the value), so I will give you examples below and I will use US as what was measured too.

For example, if a work item took whole sprint (10 days to complete), but you only spent a total of 16 hours actually working on it, then you would have a flow efficiency of 20% (rationale is a day = 8 hours).

Now a bigger example, imagine a development team working on a Scrum project with a two-week sprint duration (10 working days). The team has a Sprint Backlog consisting of various user stories and SBI, I will use US as my example here:

• US 1: It takes 2 days to start actively working on it and 2 days to complete it, with a total of 4 days.
• US 2: It takes 1 day to start actively working on it and 2 days to complete it, with a total of 3 days.
• US 3 : It takes 2 days to start actively working on it and 0 days to complete it, with a total of 2 days.
• US 4: It takes 3 days to start actively working on it and 2 days to complete it, with a total of 5 days.

Total active time spent on USs: 6 days Total time from start to end of the sprint: 10 days.

Flow efficiency for the Sprint = (6 / 10) * 100% = 60%

What does it tell us in scrum? Now compare how many other SBIs you had in a sprint, what if you had 12 items in total, so basically, 4 US and 8 of something else? (can be spikes, can be defects, can be fastlane items etc). There is a separate conversation on all types of sprint backlog citizens, yes, many of them should be represented, but in what ratio? In my example above, 30% of what brings value has a flow efficiency of 60%.

Why does it tend to be so low? This could stem from factors such as fostering specialized expertise, facing inter-team dependencies, employing limiting processes, ensuring full team utilization at all times (this is something I have seen in many of teams, when a PO has a fear or "under utilization of team members"), taking on unanticipated tasks, having an abundance of tasks in progress, and navigating context switches among others - but what is interesting is again, are these work activities that we spend time on bringing value?

Kanban Example:

Kanban is focused on improving the efficiency of the entire process flow, from idea to delivery.

Flow efficiency in Kanban measures the percentage of time a task spends being actively worked on compared to the total time it takes from the beginning to the end of the workflow.

Now, consider a Kanban board with a workflow consisting of the following stages: Backlog, In Progress, Testing, Done.

• Task 1: It spends 2 days in the Backlog, 1 day In Progress, and 1 day Testing.
• Task 2: It spends 1 day in the Backlog, 2 days In Progress, and 0 days Testing.
• Task 3: It spends 0 days in the Backlog, 2 days In Progress, and 0 days Testing.
• Task 4: It spends 2 days in the Backlog, 2 days In Progress, and 1 day Testing.

Total active time spent on tasks: 7 days Total time from start to end of the workflow: 14 days (lead time).

Flow efficiency for the Workflow = (7 / 14) * 100% = 50%

Unless you are working on one thing at a time, and you never get interrupted, your lead time equals your cycle time. Waiting time can be encountered for many reasons: dependencies, priority changes, too much work-in-progress, etc.

Stated in another way: work-in-progress isn’t always actually in progress. Flow efficiency tells us how often that is true and where exactly in kanban your flow is not optimal. In Kanban the center to calculating flow efficiency is to know how long work spends waiting.

What if we work on more items?

In Scrum, the throughput of a team (i.e., the amount of work completed within a given period, usually a sprint) can vary depending on factors such as team size, experience, complexity of tasks, and process improvements. Ideally, the goal is to achieve a sustainable pace that remains consistent over time, allowing the team to reliably deliver value in each sprint. - Should this number increase over time?

While it's possible for a team's throughput to increase over time as they become more proficient and efficient, there are several factors that can impact the sustainability of increased throughput:

1. Burnout: Rapidly increasing throughput can lead to overwork and burnout, potentially resulting in decreased productivity, quality issues, and increased employee turnover.
2. Quality vs. Quantity: Focusing solely on increasing throughput may lead to a compromise in quality, as teams may rush through tasks (and they will be "in progress" longer since they will have defects).
3. Capacity: Teams may have a natural limit to their capacity, and pushing for higher throughput beyond this limit can lead to bottlenecks, delays, and increased stress.
4. Scope Creep: Attempting to increase throughput too rapidly without proper planning and prioritization can lead to scope creep, with teams taking on more work than they can handle effectively.

So we should limit number of work we do not to have additional stress? Is there a name for it?

In both Scrum and Kanban, Work In Progress (WIP) limits serve as a means to optimize the flow of work through the system and prevent teams from taking on more work than they can handle efficiently. However, the approach to WIP limits may differ slightly between the two frameworks:

Scrum:

• In Scrum, the concept of WIP limits is less explicit compared to Kanban. The Sprint Backlog, which is a set of work items selected by the Development Team for the current Sprint, implicitly sets a WIP limit for the team (but then look at the cumulative flow diagram to see how many items were in progress same time).

Kanban:

• In Kanban, WIP limits are a foundational principle, and they are set explicitly for each stage of the workflow on the Kanban board.
• The WIP limits for each stage are determined based on the team's capacity, historical throughput (see above), and the desired flow of work (see flow efficiency). WIP limits may be adjusted regularly based on feedback and performance data to optimize flow and ensure work is not bottlenecked at any stage.

WIP Limit Changes Over Time:

• In general, the goal for both Scrum and Kanban is to maintain a stable and sustainable flow of work. Therefore, WIP limits should be adjusted only when necessary and with careful consideration of the team's capacity and performance.

In summary, the ideal approach to WIP limits is to maintain a balance between throughput and work efficiency. WIP limits should be adjusted gradually and carefully over time based on the team's capacity, performance data, and feedback from the team itself. The goal is to achieve a sustainable flow of work that maximizes productivity and minimizes bottlenecks without overwhelming the team or compromising quality.

All in all, what should you use?

1. Lead Time : The time taken for a work item to move from the beginning to the end of the workflow.

2. Cycle Time: The time taken for a work item to be completed after it starts progressing through the workflow. Cycle time measures the actual time spent working on a task and is helpful for identifying areas where improvements can be made to speed up delivery.

3. Throughput: The number of work items completed within a certain time period. Throughput indicates the team's capacity to deliver work and helps in forecasting and resource allocation.

4. Work in Progress (WIP): The number of work items currently in progress at any given time. Limiting WIP helps prevent overburdening the team and reduces multitasking, leading to smoother flow and faster delivery times.

5. Cumulative Flow Diagram (CFD): A graphical representation of work items flowing through various stages of the workflow over time. CFDs help visualize bottlenecks, identify trends, and track the overall flow of work.

6. Flow Efficiency: The ratio of value-added time to total lead time. Flow efficiency measures the percentage of time spent actively working on a task versus the time it spends waiting or blocked in the workflow. Higher flow efficiency indicates smoother flow and reduced waste in the process. - Here please note WHAT are we working on, since having high flow efficiency on something that gives no value to the user is not helping much.