There’s gold in them thar hills: how magical thinking destroys value in biotech/ pharma R&D

We’ve all been there, at some point or the other. You’re attending this sales talk — or even a scientific seminar — and the speaker opens with a slide that looks something like this:?

And at this point, the speaker will say something to the effect of “and that’s why our <insert research topic/ technology here> is more important now than ever before”.

Well, forgive me for sounding jaded, but if we keep seeing the same opening slide over and over again, then maybe it means that just because the problem has been identified correctly doesn’t mean that the solution will work.

This “déjà vu with the opening slide” problem is emblematic of a larger tendency in our industry: the pervasiveness of magical thinking. Here are seven statements you could make in any biotech or pharma meeting room — preferably in a loud and confident tone of voice — to get the audience nodding their heads in agreement:

1.??????? “Speed is of the essence”

2.??????? “Big data is the key to better decision-making”

3.??????? “The pricing and reimbursement landscape is the biggest driver of profitability”

4.??????? “Early evidence of clinical efficacy is key”

5.??????? “We can derisk our biological hypothesis by working on targets that others are also working on”

6.??????? “Precision medicine is the wave of the future”

7.??????? “Innovating with cutting-edge modalities is the best way to approach challenging indications”

“Wait, what?” you may be saying to yourself at this point. We hold these truths to be self-evident, do we not?

As it so happens, each of these statements is an example of magical thinking. These are by no means the only examples of widely held beliefs in our industry that destroy value, but this is a broad enough list to illustrate the larger point. So, let’s dig in, one cliché at a time, and see what the problem is.

?

Speed is of the essence

You will often hear it said that speed is critical for biotech/pharma business outcomes. After all, time is money — if you can get a drug to market faster, you can start generating revenue sooner, right?

Well yes, if drug R&D programs were always successful. Instead, over 90% of drug discovery and development programs fail, making for a staggeringly high combined failure rate. If you think about it, that means for every hundred programs started at the discovery stage, only one will make it to approval. So, we spend a lot of time thinking about how to maximize returns from the one successful program out of that hundred, but the other ninety nine programs also contribute significantly to the bottom line.

The therapeutic window — the range between a drug's effective dose and its toxic dose — drives any drug’s success or failure. Specific choices impact this therapeutic window (we’ll get to that in the next section), and drug candidates lacking a therapeutic window fail clinical development. Rushing to make decisions can lead to poor choices that ultimately result in failure, rendering the "speed" moot.

The Net Present Value (NPV) of a program or molecule that fails is less than zero. In business, the NPV is a crucial financial metric used to evaluate the profitability of a project. It represents the difference between the present value of cash inflows and outflows over a project's lifetime. When a molecule fails, especially after significant investment, the NPV plummets, leaving a trail of financial loss. The aggregate burden of these failures, it turns out, is the dominant driver of R&D productivity.

As a thought experiment, imagine the difference in NPV between a pipeline with two approved therapies instead of one. Even a small increase in success rates can quickly outweigh the loss of NPV that occurs due to the time value of money. A systematic model-based analysis done some years ago showed that the top factors driving productivity in Pharma were the probabilities of technical success in Phase II, Phase III and NDA submission. Our internal modeling supports this conclusion (keep an eye out for a white paper on this topic soon). Late-stage failures drive the overall productivity and ultimately the profitability of pharma and biotech, and the seeds for these late-stage failures are often laid in choices made far earlier.

While it’s true that idle time during R&D destroys value, far greater damage is done by faulty decision-making under time pressure.? The better strategy is to focus on making well-informed decisions that increase the likelihood of success, even if it means taking a bit more time.

?

Big data is the key to better decision-making

While the idea that "big data" can revolutionize decision-making in drug development is appealing, the reality is more nuanced. Drug discovery and development involves a series of critical decisions, including the choice of molecule, formulation, dose schedule, starting dose, dose escalation scheme, patient population, and clinical endpoint. Poor decision-making in these areas is a common cause of program failure. While AI and machine learning approaches are useful in some arenas of drug R&D, their utility is often overstated. (To learn more about the causes of program failure, see this article of mine on “Pitfalls in Drug Development ”).

For example, patient selection via machine learning has been attempted for over 25 years, yet most successes in this area have come from retrospective analyses of smaller clinical datasets, rather than from ML models trained prospectively on big datasets. That said, AI has made a meaningful impact in other aspects of pharma R&D, particularly in supporting and extending virtual screening techniques at the discovery stage.

AI works best when there is an enormous body of training data that can be leveraged as a feature vector to predict the outcome. In particular, the training data must comprise sufficient instances to inform how the features may or may not predict outcome, and the underlying mechanism must support a basis for prediction. This is why LLMs perform so well in mimicking human speech and why no AI model will ever be able to predict a person’s lifespan or income from the positions of the stars on the day they were born. AI models in drug discovery and development are based on data that may or may not predict outcome, and they are often subject to overfitting. (To learn more about the ways in which AI and other mathematical techniques can go wrong in drug R&D, see my article Pitfalls in Mathematical Modeling in Pharma R&D ).

In the end, big data can enhance decision-making in some contexts, but it’s not a panacea, especially in a field as complex as drug development. It's always worth asking the questions "is this a question that's best answered using big data?" and "was the method implemented correctly?" when using big data-derived predictions for decision-making.

?

The pricing and reimbursement landscape is the biggest driver of profitability

Contrary to popular belief, the biggest driver of profitability in drug development is not the pricing and reimbursement landscape — it's clinical failure . As mentioned above, the vast majority of pharma R&D programs fail. It’s tempting to “plan for success” -- to tweak the target product profile by including features that would favor marketability, all other factors being equal. However, this approach can be counter-productive because development choices impact the likelihood of program success in the first place. What would you rather have as an outcome for your program- an approved i.v. drug on a once-a-week schedule, or a failed program that attempted to develop an i.v. drug on a once-every-three-weeks schedule? There are many examples of where an aspirational schedule choice drove a program to failure.

For example, a recent paper published by us elaborated on the post-marketing failure of Mylotarg, an antibody-drug conjugate (ADC) for acute myeloid leukemia. Mylotarg proved toxic on its approved, twice-monthly schedule and was withdrawn from the market until a less convenient but better-tolerated fractionated schedule was chosen. We showed in our modeling that other schedules may have an even broader therapeutic window. This example illustrates the risks of allowing commercial aspirations—such as a more convenient dosing schedule—to dictate R&D strategy.

More broadly, perceptions of pricing and reimbursement based on historical experience can be misleading. Drugs are like houses, they sell for what they sell for, and Zillow only gets you so far! For example, the market for COVID-19 vaccines was unprecedented – Pfizer made $57 billion in 2022 alone with their vaccine. As another example, consider the Gleevec case. Before Gleevec, chronic myeloid leukemia (CML) ran its course quickly – only 30% of newly diagnosed patients made it to five years. Gleevec's success dramatically increased survival rates, which in turn meant there were many more people living with CML now. As a result, Gleevec’s market potential turned out to be far larger than was initially projected.

The key takeaway here is that allowing commercial aspirations to dictate R&D strategy can easily take you down the wrong path. A drug's performance ultimately determines its profitability, not market projections. Focus on discovering and developing the best molecule you can (hint: it's the one with the widest therapeutic window). Then, build the commercial strategy to compellingly articulate the value proposition of your drug. Locking in choices early on that make marketing "easier" at the expense of the therapeutic window can back a program into a corner.

?

Early evidence of clinical efficacy is key

The thing about "early evidence” of clinical efficacy is that it’s an oxymoron. If it’s early, it’s not evidence. Evidence of efficacy can be framed in two ways – does my drug work? And does it work better than the competition? The first question is precisely what Phase II trials are designed to examine, and the second is what Phase III trials are designed for. Both these types of trials often enroll hundreds or even thousands of patients. Relying on early clinical “signals” as definitive proof of efficacy from smaller Phase I trials can often lead to poor decision-making.

It’s not uncommon to see situations like this (using an Oncology Phase I all-comers trial with different cancer types as an example): the first response is observed in a patient with lung cancer, leading to more lung cancer patients being enrolled in the trial and eventually a Phase II in lung cancer. Similarly, a Phase I trial with two different schedules may see slightly higher numbers of responses in on an every-two-weeks schedule versus a weekly schedule. Even though the differences are not statistically significant (and the numbers of patients on each arm is different), the team will often decide to go with the every-two-weeks schedule, because of the clinical “signal”.

The thing is, comparing percentages — and ratios — with relatively small numbers can be deceiving, a term referred to as “the tyranny of small numbers.” Making conclusions (or developing biases) based on small samples is a risky game. Further, early signs of efficacy might not hold up in larger, more diverse patient populations, and in some indications (such as Oncology) the clinical endpoint changes from Phase I/II to Phase III. Additionally, a lack of randomization, lack of blinding and short follow-up periods can all lead to misleading conclusions. Thus, basing critical development decisions on early clinical efficacy “signals” can lead to costly failures in later stages.

The focus should be on designing robust clinical trials that can provide conclusive answers, rather than rushing to interpret early data. In particular, developing a “pharmacological audit trail ” linking pharmacokinetics (PK), pharmacodynamics (PD) and efficacy from early-clinical data is crucial for supporting rational decision-making. Such an audit trail lets you evaluate drug performance in terms of two fundamental questions: is it getting there? And is it doing anything?

?

We can derisk our biological hypothesis by working on targets that others are also working on

From an epistemological perspective, the validity of this proposition hinges on the extent to which a competitor’s failure yields useful information. The reality is that they might not know why their program failed either! The ability to mine competitor data for insights that can drive decisions is highly limited, particularly when there is an "n of 1" situation with multiple competing explanations.

When a competitor's program fails, the information you can extract may be minimal or even misleading, leading to wrong conclusions and decisions. When a competitor’s drug for a novel target fails to achieve disease modification, there are two competing explanations -- either the drug failed to inhibit the target at the site of action, or the drug succeeded at inhibiting the target, in which case inhibiting the target does not lead to disease modification. Knowing which of these causes was responsible for the competitor’s failure is crucial to extracting meaningful information from the outcome. Unfortunately, this requires constructing a PK-PD-efficacy relationship from the competitor’s data, which is almost never shared with enough granularity to make that possible.

On the other hand, when a competitor succeeds, it can reduce the potential upside for your project, as now you’re a late entrant in the market and have lost your first-in-class advantage.

Herding— when multiple companies chase the same targets— is common in the industry, but it's not necessarily a smart strategy. Despite its prevalence and long-running popularity, there is no evidence to suggest that herding is a good strategy. Instead — as several authors have pointed out — it often leads to wasted resources and reduced returns.

?

Precision medicine is the wave of the future

It’s been the wave of the future for a couple of decades now. The appeal is somewhat intuitive — to quote a Harvard Business Review article on the topic from 2007: “These developments have vastly expanded doctors’ power to customize therapy, maximizing the effectiveness of drug treatments and minimizing their side effects... Accelerating the adoption of personalized medicine is enormously important in terms of saving both lives and dollars”. Unfortunately, the vision of the future presented as a fait accompli in that article from way back when is still far from being reality. The concept of matching a patient's genetic profile to a specific treatment is appealing, but it has proven challenging in practice. Two decades since Francis Collins first pitched the vision for precision medicine, only 8-15% of Oncology patients are eligible for personalized medicine. Of those who are eligible for these treatments, most will fail to respond. For example, a study published in Cancer Discovery found that only 7% of enrolled patients who had been matched on a molecular basis to a precision medicine agent responded to it – other studies have shown similarly dismal results. So, 7% of 15% is 1%. Two decades on, that’s far from revolutionizing cancer care. While proponents of precision medicine have argued that we need to stay the course, a case can be made that the foundational assumptions of precision medicine are unsound .

On the other hand, one area of personalization that has seen success is tailoring drug dosing schedules to patient pharmacokinetics. This approach (known as therapeutic drug monitoring, or TDM ) has been powerful and valuable for certain classes of drugs, particularly for those with a narrow therapeutic window.

Personalization is a worthwhile rescue strategy, but relying on it too early in the drug development process can divert attention from the fundamental work that needs to be done in preclinical and clinical pharmacology. Identifying the right dose, schedule, and route of administration is crucial, and this requires rigorous, methodical research. While personalized medicine may be able to command a price premium based on a “trendier” value proposition, that price premium may not translate into increased revenue because of a smaller addressable market. If personalization doesn’t actually lead to a wider therapeutic window, it’s unlikely to improve profitability or patient outcomes.

The reality is that, for many diseases, we simply don't yet have the knowledge or tools to effectively personalize treatment on a large scale. Until we do, it's important not to let the allure of personalization distract from the fundamental work of developing robustly effective therapies.

Innovating with cutting-edge modalities is the best way to approach challenging indications

Innovation is critical in drug development, but the idea that cutting-edge modalities are the best way to tackle challenging indications is not always true. Complex modalities, such as multi-specific antibodies or gene therapies, often introduce new layers of complexity into the development process. This complexity can make it more difficult to control outcomes, potentially leading to unforeseen challenges and failures.

For example, advanced modalities like CAR-T therapies have shown remarkable results in some blood cancers but have faced significant challenges in solid tumors due to the complexities of targeting and delivery. Similarly, the development of bispecific antibodies or other multi-functional biologicals introduces additional layers of complexity that can make them difficult to manufacture, test, and ultimately bring to market. The risk here is that the novelty of the modality overshadows the importance of understanding the basic biology and pharmacology of the disease.

A better approach is to let the choice of modality be driven by the specific needs of the indication. A simpler, more traditional approach might be more effective in some cases, while more powerful modalities such as ADCs and bispecific antibodies may be the best option in others. The key is to match the modality's strengths to the indications where it will be most decisive for the outcome, rather than pursuing innovation for its own sake. If you’re looking for more information on this topic, here is a white paper of ours where we show how this can be done.

There are better ways to approach these issues

At its heart, pharma R&D is an engineering endeavor. Taking a systematic and pragmatic mindset can go a long way towards reducing the risks posed by magical thinking. At each step along the way, it’s important to understand clearly what data supports the choices being made. Here are some general principles that can be helpful:

Be open-minded about choice of indication – it's a whole world of unmet medical needs out there. Many common diseases, like endometriosis, have no effective treatments. Other examples include certain rare diseases that have yet to see significant therapeutic advancements. By broadening the scope of research, companies can address these unmet needs while reducing competition and increasing the chances of success.

Don’t let market considerations dictate the Target Product Profile (TPP). This assumes that drugs always work, but pharma R&D is not the same as sneaker design. Make the drug work first. Once you have a drug that works well, approach the branding, pricing, and value proposition in a careful and scientific way. First, build the best molecule you can. Then, build the best commercial strategy you can to compellingly articulate a value proposition and capture it. When doing this, your job will be made easier if the approved drug has a compelling clinical benefit. Forcing choices on the project team that make marketing "easier" at the expense of the therapeutic window will ultimately backfire.

Be skeptical about new technologies that promise an immediate revolution in drug discovery and development. Given that dartboard decision-making is a large part of the problem, a focus on carefully thought-out decision-making for key choices is crucial. Introducing brand-new technologies adds complexity and risks during the already challenging development process. It’s like flying a plane — separate the building phase from the flying phase for better results. An Occam’s razor approach to modalities is essential: avoid excessive complexity. For example, two-binder peptides and trispecific antibodies may seem innovative, but they may also introduce unnecessary risks. Instead, match the modality’s strengths to indications where it will be decisive for the outcome. Don’t treat a skin condition with an ADC, for example. Balancing the risks of the modality and the biology is crucial — don’t double up on your risks by treating a previously unaddressed disease with a brand-new modality! (Just because it worked for Moderna, doesn't mean it's going to work for you.)

Focus on the therapeutic window – The therapeutic window is a key determinant of profitability, as it both contributes to the risk of failure and impacts marketing leverage. A highly efficacious drug will find it easy to command a pricing premium, all other factors being equal. Conversely, a drug candidate lacking a therapeutic window will never see the light of day.

Focus on the translatability of models being used in decision-making. Both the model and the translational framework need to be thought through carefully. For many disease indications, the preclinical or animal models have not been validated (meaning that they have not been shown to predict clinical efficacy). ?For indications such as these, build a strategy to support clinical development using a PK/PD/efficacy “audit trail” which I described earlier. Identify the largest risk that cannot be mitigated until you reach the clinic (e.g., will this modality work? will inhibiting this target provide a therapeutic window?) and design your development strategy and individual clinical trials to derisk this proposition as efficiently as possible.

o Unmitigated biology risk: Recognize that you will not be able to prove that your target inhibition will modulate the disease with preclinical or mathematical models. The time to prove this proposition is in the clinic. Design your biomarker strategy around efficiently proving this, focusing on linking target engagement to efficacy. Establishing the PD--> efficacy link as early as possible in the clinical setting is key to derisking programs like this.?

o Unmitigated modality risk: Design your biomarker and pharmacokinetics (PK) strategy in early clinical trials around derisking the modality, focusing on drug delivery to the compartment and target engagement. Build a preclinical package that is focused on supporting the establishment of a PK-->PD relationship as early as possible.

These tenets can help you with the specific examples of magical thinking that we discussed in the previous section, but they go beyond that. Taken together, these principles can form the basis of a pragmatic and engineering-focused approach to drug discovery and development that avoids the trap of hype and focuses on enabling decision-making that creates, rather than destroys, value.

That leaves a bigger question unanswered, though.

Deconstructing magical thinking

As I mentioned earlier, those seven statements above are far from the only examples of magical thinking in pharma R&D. Widely held beliefs that lack evidence to support them are unfortunately quite common. There’s a larger ‘meta’ question here — how do these beliefs come to be widely held?

At its heart, all magical thinking arises from one of a few cognitive traps:

Groupthink: In any collaborative environment, there is a natural tendency to align with the majority view to avoid conflict. This can be true even if the group in question has beliefs that don’t align with reality, a phenomenon known as the firehouse effect . Team members’ discomfort with appearing contrarian can lead teams to avoid examining the factual basis for their assumptions. This allows the proliferation of beliefs that are not supported by the facts, such as the ones in this article, as well as magical thinking that may be specific to the project team.

An excessive faith in technology: The belief that technology alone can solve complex problems is misguided. As an analogy, the Nazi regime during World War II placed great faith in "Wunderwaffen " (wonder weapons) — jet fighters, cruise missiles, and ballistic missiles were all deployed and used first by the Nazis. However, these "superweapons" afforded them no meaningful tactical or strategic advantages, and in some cases contributed directly to defeats . Cutting edge technologies often come with their own untested scientific hypothesis and operational teething troubles, and market perceptions around these technologies can be fickle . Having to iron out the kinks in brand new technology (or a modality) contributes risk to a program at several different levels.

Lack of deference to expertise: The expert for a given project is almost always the line function representative. Usually, the biologist on a project team will be the first to say that the biology is “more complicated than that”, the toxicologist will take a contrarian view, and the clinician will point out that “I don’t think that'll work for enrolment”. Line function representatives in many companies get overruled, either in a line or matrix context. If you’re flying a plane, and the mechanic tells you that the engine sounds funny, listen to them. It could make the difference between getting home tonight and turning into a smoking hole in the ground. (If you’re a senior leader reading this, it’s worth remembering the space shuttle Challenger disaster, a tragedy then and a parable now . Creating a culture where your direct reports can disagree is absolutely critical. Resist the urge to overrule your team’s technical calls. It’s precisely when the feedback from your team is frustrating that you have the most to learn).

A lack of a sense of agency: This is the belief that individual actions don't matter because the outcome is predetermined. Teams where the line function experts are frequently overruled on key decisions may adopt these beliefs more readily. In drug discovery and development, this mindset can lead to a lack of willingness to "stick one’s neck out” and missed opportunities to make critical course corrections. ?In reality, every decision, no matter how small, can influence the trajectory of a project. The collective impact of these decisions often determines the success or failure of a program.

The belief that value creation can be separated from the end outcome of the molecule: The idea that value can be created independently of the drug's clinical success is a fallacy. Ultimately, the value of a drug is tied to its performance in patients. If a development strategy doesn't align with this reality, it will fail. Some believe that they can create value through clever marketing or strategic partnerships, regardless of the drug's performance. However, the reality is that a drug's value is intrinsically tied to its clinical outcomes, and financial outcomes are often anchored directly to drug sales.

Viewed in this light, then, magical thinking is really a set of faulty heuristics that teams rely on for decision-making under pressure. They are a natural consequence of operating in an environment where the stakes are high, and the connections between one’s choices and outcomes are not always immediately apparent. Let’s dig a little more into the link between magical thinking heuristics and the destruction of value in a pharma pipeline.

?

?

How does magical thinking destroy value?

Magical thinking leads to short-term decisions that often have long-term negative consequences:

-?Everyone works on the same diseases and targets: This creates overcrowded areas of research with reduced potential for success because of intense competition.

-?Effort is misallocated: Project choices lead to taking on unnecessary complexity or investing in technologies that don’t create value.

-?Rushed decisions lead to mistakes: Mistakes are the main driver of failure in drug development, and they are often the result of decisions made under false deadlines and without sufficient information.

Magical thinking constrains options and forces decisions on an arbitrary timeline with unnecessarily limited information. A false sense of urgency and artificial deadlines drive dartboard decision-making , where critical choices are made without sufficient data or analysis. There are decision-making frameworks that can be used effectively in an information-poor environment, such as decision science and Bayesian statistics. However, all too often, critical-path decisions made under time pressure are often based on gut feelings or assumptions. The mentality becomes "I don’t need to be smart; I just need to be lucky." But luck is not a strategy. It’s why people go to Vegas and come back broke. In the end, the house always wins.

Pursuing drug discovery and development strategies based on magical thinking alone is not uncommon in our industry. Often, companies and departments will ride the hype cycle for several years before an abrupt turnaround of fortunes:

-?Deal values collapse for candidate drug molecules that have been outlicensed, as contingent payments for a future date are usually linked to outcomes. It is common to see companies take a large write-down on glossy acquisitions of late-stage assets.

-?Investors lose patience after a large enough number of clinical setbacks in the field.

-?Stock prices tank for publicly traded companies built on a scientific premise that fails to pan out.

-?Departments in large pharmas are cut if the underlying scientific premise fails to deliver on time.

The consequences of magical thinking can often take time to play out, but in each case, the gap between what was promised is reflected in negative business outcomes. The tendency to ‘herd’ into high-risk, high-return strategies (underpinned by magical thinking) has been shown by others to play a large contributing role in the industry’s productivity crisis in recent decades.

?

How to break the spell of magical thinking

As we discussed earlier in the article, magical thinking is commonplace in our industry. Ironically, companies in industries with a far lower concentration of scientists (such as retail and consumer goods) often take a far more scientific approach to developing business strategies and differentiating themselves from their competition.

Magical thinking is the exact analog of the statement “It is widely known that” in a paper — usually, that framing is reserved for beliefs that the author is unable to find a peer-reviewed reference to support. Sometimes beliefs like that are correct, and other times they’re just wrong.

The antidote to magical thinking is a spirit of inquiry. A spirit of inquiry demands that we question assumptions, ask for evidence, and be open to changing our minds when presented with new information. Two hallmarks of statements made from a standpoint of magical thinking are that they are often widely regarded as true and invariably sound reasonable. When you encounter a statement that you suspect may be magical, consider saying, “oh really” (in your head) and “that’s interesting, could you share more about the data they used to draw that conclusion (out loud)?” ?By doing so, you can gently challenge assumptions and encourage a more rigorous examination of the facts underlying a particular belief.

Recognizing the cognitive traps that underlie magical thinking and adopting a more rigorous, evidence-based approach to decision-making can help companies better navigate the complex challenges of drug development.

One approach that can help in this regard — in keeping with an engineering mindset — is to use model-based approaches to support critical path decisions, placing major decisions in a hypothesis testing framework. Adopting model-based approaches won’t immediately rid you of the problem of magical thinking, as organizational culture is usually one of the biggest drivers. That said, model-based approaches are particularly useful in helping teams examine underlying assumptions and explicitly evaluate tradeoffs when choices need to be made.

Fractal Therapeutics is committed to helping companies avoid these pitfalls by providing model-based drug discovery and development services that prioritize careful, informed decision-making. Our focus is providing rigorous and thoughtful strategic input, coupled with modeling and pharmacology support as needed to help our clients maximize their chances of bringing effective, safe therapies to patients who need them. Let's break the spell of magical thinking together and embrace a future where data and facts, not popularly held beliefs, drive our decisions.