Why capacity models matter

This post is a sequel to the Method Matters Podcast with Carlos Taborda from July 2019 (refer to 32:00)

A capacity model is a continuously refined artifact that helps to forecast how much work can get done in a given time. At Agilent’s Software and Informatics Division, our capacity model helped us to more than double our productivity over a couple of years. It also works every release to help our stakeholders and customers understand what to expect from us. Our capacity model is at the foundation of our team’s confidence and our customers‘ trust.

Our team does Agile Scrum software development on a global scale. In Agile development, teams work in short cycles called sprints. At the end of each sprint, the team must deliver working software that a customer or stakeholder can evaluate. To do that, they work with a customer or a proxy to write requirements in the form of stories that can be completed, tested, and demonstrated within a single sprint. To do that, they estimate how much effort the story needs so it can be sized to fit a single sprint. This depends in turn on a solid “definition of done” - the set of conditions that must be met for a story to be considered complete. We learned that a good Definition of Done is critical to success.?More on this later.

To simplify estimation (and keep ourselves honest), we dedicate developers and testers to each scrum team (we target 6 dev and 3 testers/test developers), plus a scrum master and a product owner who share two scrum teams. We use a fixed two-week sprint cycle. We learned the hard way to define a “point” of effort as being equivalent to a raw staff day. A reproducible unit of work means we can aggregate the capacity of several teams into an estimate of total capacity. We deliver software to market on a fixed release cadence, and give first priority to quality (both freedom from defects and fitness for use), second to hitting our cadence, and third to scope. We design our releases so that scope can be ejected if we run into trouble.?

At our scale, it was critical to have a single?system of record for all work and estimates (we use Atlassian Jira). This lets us assign work flexibly among teams and generates useful reports to reality-check our capacity model and our progress toward a release.

Before we had a capacity model, we planned releases from a ranked backlog of work we wanted to be done, but found that less work fit our release at the start of development than we had hoped. We were over-planning relative to what we could get done, and disappointing our stakeholders. We made our first capacity model in response to this challenge. Over time, this has resulted in reduced planning effort and much more realistic expectations about what would fit in a release.

Our first capacity model accounted for estimation errors, defect fixing time, time for customer emergencies, and time to maintain prior releases.?Subtract this from “loaded” staff-days in the release and we had an estimate of capacity for?stories delivered to market (“payload”), including development and testing.

To estimate loaded staff-days we relied on a “focus factor” that estimates how much of a developer’s day can actually be used for developing, as opposed to other activities of corporate life. Note: Special thanks to CPrime, our Agile coaches, for introducing us to this idea. We treat Focus Factor as improvable; we look for ways to grow it to improve the workplace for developers and testers and increase payload capacity.

We refine our capacity model after each release as part of our continuous improvement efforts, using Jira to review how we spent capacity in the release. We identify un-modeled capacity leaks and set realistic goals for improving them over time. We also set goals for activities that can lead to improving focus factor. Because we value trust and autonomy, we do not measure the focus factor individually. Instead, we infer it from aggregate performance and work to improve it based on the evidence we have.?

Over time we refined our model to include the capacity required to scale the team (training and hiring), capacity reserve for release planning, time lost to personal emergencies and attrition, and reserves to support cycle time targets for bug fixes. Experience also led us to refine our Definition of Done in response to situations where teams felt pressured to close out stories to be “on track” while pushing defects and technical debt later in the release. In our case, we added the expectation that stories have architectural review and code has peer review, reviewed unit-tests exist and pass, and that all critical and serious defects against the story are closed before a story can be marked “done”. It’s critical to develop a culture that rewards honesty and early action when problems arise. Ultimately it is people and culture, not rules, that create great products and services.

Two more critical learnings from our experience:?

1. Do just enough planning. Continuous grooming and high-level rough estimations give us the evidence we need to minimize over-planning waste, especially important because?Product Owner capacity is critical and difficult to scale quickly.?Product Owners operate?at three levels: portfolio, product, and release backlog.?If there is not enough of the right kind of capacity reserved for laying the tracks (architecture and longer-term scope) the release trains will slow down and start to stutter.
2. Assess technical risk and architectural changes early in each release cycle, then work the risk down early in the release. Otherwise, you will get late surprises and it will be too late to eject scope when the crunch hits.?Minimize Work in Process for the same reason - stories should be finishing all through the release rather than piled up at the end so that test load is relatively level and risk is reduced early.

Based on our experience, a capacity model can be useful at any scale, individual to enterprise, and applies to any kind of work, although it is especially good for work done with Agile methods because of their transparency.?It enables us to answer planning questions, to focus on those things that have a fighting chance of getting built in the next release, and to reflect the real costs of change and waste. With our capacity model, we can confidently predict what we can achieve, how fast we can scale, and improve every time.

If you’d like to give capacity modeling a try, here’s what you need:?

• One system of record for stories, estimates, and bugs (Jira for example, but for personal work it could be a spreadsheet, sticky notes, or a paper list)
• A reproducible unit of estimation (we define a point as a raw staff day)
• A basic capacity model that you can refine from release to release
• An event where the team reflects available feature capacity to stakeholders so that the right amount of backlog gets groomed into each release
• An event to review and improve the model after each release
• A rigorous “Definition of Done” to prevent capacity leaks
• A defined release cadence - this can range widely from hours to months depending on the nature of your business and the maturity of your processes and build systems. It’s critically important that the cadence is fixed, and that scope is treated as the dependent variable.