What does good look like, and the problem with metrics.

Organisations need to know what is going on. This is nothing new.

Something else that isn't new is that there are many layers to an organisation, and many lenses that people use when they want to see how well things are going. What is more new is that people are beginning to recognise that the world in which we build products and serve customers is larger than just the product.

To quote Kevlin Henney :

Development needs to go further than the technical stack; the full stack includes the world and people around the software.

This is a worthy note and the sense of inclusion and scale is admirable, but it brings its own challenges. If we are to consider a variety of players in our product development enterprise, then what does good look like to them?

To a project manager; good means something done within time, cost and quality. To a product manager; good means an intuitive product that meets the needs of its target market. To a development manager; good means low bugs, easily deployable and fits the current landscape. To a developer; good means well written, well structured code and an unbroken build. To an organisation overall; good has a staggering amount of dimensions and would typically include things like improving shareholder value, high staff morale, sensible investment strategy, increasing market share, high customer satisfaction, ethical commercial reputation etc. What about what good means to a customer? (ask Theodore Levitt perhaps)

This is potentially an intimidating amount of information to process, and trying to collect and consume all that data into something meaningful (a dashboard perhaps) often means that the true problems or opportunities are not surfaced or understood properly.

The first reason for this is that senior leaders traditionally want to manage by the perceived best solid metric; Cash. Be it turnover, ROI, revenue, till takings, salaries or expenditures its all easy to measure and easy to read.

The next best set are solid output numbers; customer volume, footfall, transaction volume, headcount, stock inventory etc. Again all easy to measure and easy to read. But these are all reactive measures and after the fact - you've already launched your product.

And how many of these things are actually just vanity metrics?

But how do you measure whether you are making the right choices for product development before committing to developing it? Or that the organisation is efficient in its endeavours, that product quality will be high, or that the overall goals of the organisation as a whole are being met. Most of these can not be measured at all by measures after the fact, certainly not by the ones in the last paragraph.

There are worthy attempts at taming this beast going on. We've had OKR's (Objectives & Key Results) rattling round for decades which have become more popular since John Doerr published his book "Measure What Matters". We had "earned value management" frameworks back in the day, we had "project accounting", we've had Lean Startup elements including A/B testing and Cohort Analysis, and also new things like "Target Steering" from WoW! Agile as well.

What do you use to solve this problem, and at what level of the enterprise? ..and does it really solve it? How can we actually predict the future in a meaningful enough way to be able to work towards it and be successful?