Is the Distinction Between Outcomes and Output Overdone?

In about half the conversations about agile, I hear someone make the point that “teams should focus on outcomes rather than outputs.”

It’s gotten to the point that I’m tired of hearing it, especially because I somewhat disagree with it.

I know that’s blasphemous to say, but hear me out. To start, let’s be clear about the difference between the two words.

An outcome is a result. It’s something a team or product has achieved. An output is something produced along the way. The output isn’t always important in and of itself, but it leads to the outcome.

An Example of an Outcome and Output

A common example involves the effort by the Gates Foundation to eradicate malaria.

The Gates foundation could measure the outcome of their efforts by looking at the number of people killed by malaria in Sub-Saharan Africa and South Asia, the two regions with the most malaria cases.

Alternatively, they could measure the output by looking at how many bed nets they distribute. Bed nets treated with insecticide is one of their strategies for preventing bites by infected mosquitoes at night while people sleep.

The argument is that the Gates Foundation should measure the reduction in malaria deaths rather than how many bed nets they distribute. After all, if the goal were to deliver bed nets, the Gates Foundation could do so most effectively by distributing them near their headquarters in Seattle. Seattle isn’t a hotbed for malaria but the bed nets could be distributed there more easily than halfway around the world where they’re more needed.

The Fallacy

The fallacy with arguments like this has to do with when outcome and output data becomes available. To see if they’ve achieved an outcome of reducing annual malaria deaths, Bill and Melinda Gates need to wait until the end of the year.

On the other hand, they can get data on their output (number of bed nets distributed) daily.

In other words, outcomes are a lagging indicator whereas outputs are a leading indicator. Often the best way of achieving a goal is to focus on the intermediate goals that lead to the ultimate goal.

Suppose your team has chosen to improve the quality of its delivered products. They set an outcome goal of a 50% reduction in post-release defects as measured in the first 30 days after a release.

That’s a great goal. It’s an outcome. But when can the team measure it? Not until 30 days after release.

So the solution might be an output goal. Perhaps the team decides to double the number of automated tests written each iteration.

Do automated tests guarantee a decrease in post-release defect rate? Definitely not. But unless deliberately misused, they can certainly be a good leading indicator that the team is headed toward its outcome goal. And the information is available more quickly.

But Can’t an Output Measurement Be Misused?

Arguments against measuring outputs often center around the potential for misuse. And while that potential is real, my experience is that it is rare for a team to deliberately mislead by gaming a leading indicator when there is an associated lagging indicator that will be measurable in a few months.

The Gates Foundation is not, for example, going to proclaim total victory over malaria because they distribute a billion bed nets in Washington (a misleading leading indicator) if they know a verifiable lagging indicator (deaths) will be available in a few months.

Outputs Lead, Outcomes Lag

Because outputs are leading indicators, they serve a complementary role to outcomes, which are lagging measures. This means teams often need to look at both.

Yes, the outcome is the more important measure. It is how we will ultimately evaluate the success or failure of a project. But well-defined and well-considered outputs can usually be identified that can tell us in a timely way whether we’re on track to achieve the desired outcome.

What Do You Think?

What do you think? Do properly designed output measures support achieving the ultimate outcome goals? Please share your thoughts where this post was originally published, on the Mountain Goat Software blog.

If you liked this article, you can sign up to receive my one best, short tip about agile each week. These tips aren't online and signing up is the only way to receive them.