Who puts the product in the productivity?

The words 'product' and 'productivity' are linked, but they are not the same. 

Both words come from the Latin word 'producere' which means 'to bring forth’. The word 'product' refers to something that is produced or brought forth, while 'productivity' refers to the quality or rate of producing something.

With a nod to my earlier piece, Who Puts The T in the TPP?, it is worth examining the way R&D ‘Productivity’ is measured or assessed.

The classic definition of ‘productivity’ does allow for a rate assessment, but it does get a bit vague on the ‘of what?’. Because the rate of producing approved products is so low in pharma - Novartis’ run of 14 approved products in 5 years through 2020 was an industry record; in many years it is possible for major pharma companies to launch no new drugs at all - there is a tendency to look for something other than that singular outcome to measure.

If I were evaluating a simple conveyor belt assembly line, and I could measure the rate at which the four key components were assembled, and the rate remained consistent, I could confidently predict the new rate of productivity if a positive change was made to all four steps. If, however, it didn’t matter how fast the first was produced, because step 4 took the time it took, and was unimproveable, my measurement of ‘productivity’ would have to focus only on the one step - steps 1 to 3 are independent of the overall rate. They’re critical, but of no value to someone looking to improve the factory’s ‘productivity’.

In the ‘old’ or traditional linear model of phase I, phase II, phase III and so on, it is traditional to see rates of phase transition as linked - that ‘success’ at each phase is predictive of overall success, like an assembly line. Because the final number (the rate of approvals) is so low and so distal, it feels better to measure inputs - things that happen in a visible timeframe. However, we know that phase I ‘success’ is not predictive of future success - somewhere south of 8%, however you measure it. Like the conveyor belt, you could increase your phase I to phase II transition rate to 100% and make zero difference to your overall output. This thought experiment would make a company’s ‘productivity’ look tremendous. Some companies have proudly averaged out their phase I ‘success’ rate, their phase II ‘success’ rate and so on, and claimed that they’re better than average, at 2% - so you could artificially improve things by making phase I easier, with no impact on your late stage pipeline.

So, the steps that are traditionally measured relate very badly to the goal of approval - we know that more drugs that are ‘successful’ at phase I do not become approved products, and so on throughout development. Drugs that have successful phase IIIs do have a positive correlation with approval, but it is not 100% (perhaps, in the current model, if there is one number you want to be 100%, it should be this one). So, perhaps, it is wrong to regard these as useful surrogates in any way. As Jack Scannell’s work has shown, it is possible and desirable to drastically improve predictive validity - but we still have to focus on the end goal. Perhaps pharma needs to see itself more as a production line than an assembly line.

This lack of flow-through from early phase is why many companies now choose to focus on acquiring products that are phase III-ready, or the companies who have them.

To further complicate matters, on a subject my blog has focused upon, even approvals are poorly predictive of on-market success. It is entirely possible that we have built an industry where incentives are focused on surrogates that do not predict, and on outcomes that do not matter, in terms of return on invention (or return on capital). It is possible we’ve built a conveyor belt, applied all of the kaizen possible, but failed to note it points towards only a recycling bin. There is no product in development less useful than one that does not make it to patients. True ‘productivity’ measures should align with a singular goal - get products to patients.

The issue with the belief in a linear process is that it places all of the emphasis on the first decision, early in phase - all of the work from then is confirmatory, not exploratory - a product ‘not dying’ is seen as evidence to progress, a ‘signal’ only to those who really want to hear it. Surrogates make sense in a linear process, but they do not make as much sense in the ‘explore and exploit’ model pharma should follow. As Ed Catmull wrote in Creativity, Inc., it is possible that companies end up feeding their own ‘hungry beast’, instead of making successful movies.

The emphasis in R&D or Pharma ‘productivity’ has to be on bringing forth ‘product’ - the ‘thing that is produced’ has to be something that our customers want. There is a clue in the ‘quality or the rate’ part of the definition - pharma can measure quality against a final outcome like 'more products to patients', and put more product in productivity.