Perfect Way to Innovate vs. Almost Perfect Way to Innovate

Finally, there seems to be some insight into the CDC fiasco over COVID-19 testing. New York Times reports that CDC's kits were contaminated because of sloppy manufacturing. Apparently, the FDA inspector found:

"an astonishing lack of expertise in commercial manufacturing and learned that nobody was in charge of the entire process…. Problems ranged from researchers entering and exiting the coronavirus laboratories without changing their coats, to test ingredients being assembled in the same room where researchers were working on positive coronavirus samples…. Those practices made the tests sent to public health labs unusable because they were contaminated with the coronavirus, and produced some inconclusive results."

How could this happen? CDC is a the top public health organization in the world, full stop. They have the best minds in the world when it comes to infectious diseases, full stop.

To me, this was almost predictable. Commercial organizations are regulated and inspected by the FDA. While we complain about the oversight, it forces us to be disciplined. When entities are exempt from oversight for things like this, you would expect this to happen. It's almost a certainty, in fact.

You see, manufacturing things like drugs, diagnostics, and semiconductors is a very unnatural endeavor. Crazy endeavor, in fact. You have to have 0.00001% tolerance for mistakes. You bring down particulate matter in the air from 35 million particles per cubic meter down to 35 or less. And you test that you've done that. And you test the test to make sure that your test is working. And then you retest. You double or triple glove. And train people how to glove. And train the trainer. And test the training. And you do that for every single thing you do, every process, every raw material.

The fact is, we humans didn't evolve to do things like this. When we were living 500,000 years ago, there were no zero-failure things we had to do over and over thousands of times. Certainly there were no village tasks that required precision like we need for bunny suit manufacturing. We are not wired to act to insure 0.00001% failure.

It's hard for us to even distinguish between 0.01% risk and 0.00001% risk. Our minds don't work that way. We see it as the same thing. As a result, we naturally regress to the mean. Forget to put on that third glove? I already have two sterile gloves on both hands, what's the big deal? You were trained to put on gloves by someone who's not certified to train you how to put on gloves? What's the big deal?

In fact, the difference between 0.01% and 0.00001% is 1,000X. It is an enormous difference. Instead of having 1 contamination per year, let's say, you have 1,000 contaminations per year. This inability to distinguish between very small numbers that seem the same but aren't is similar to the fat tail problem that Taleb describes in his books. It's a cognitive blind spot.

Especially for things like manufacturing, keeping financial records, or other highly detail-oriented things, where you need a low failure rate, the default for most organization is to regress to… sloppiness. Or maybe I should be fair, it's hard to maintain 99.9999% perfection, the natural tendency is to regress to maybe 99% perfection. You need periodic audits or third party review, constant pressure if you want to maintain tip top shape.

This is why for mission-critical things, you need constant, active push toward keeping the standards high. If not, the natural tendency is to go toward disorder, higher entropy.

Now, how does this relate to startups?

Ever notice how so many successful startups are led by people who people say are perfectionists? People who are often called unreasonable? Steve Jobs. Elon Musk. Evan Spiegel. Mark Zuckerberg. Or going back a bit, Henry Ford. John Rockefeller. They are legendary for being fanatical and particular. Henry Ford scrutinized the cost of goods like a hawk. Rockefeller counted the number of weld spots on oil drums to save the cost of one or two welds. They acted as if a tiny little thing can matter.

The thing is, the tiny little things can matter. The difference between Snapchat and hundred other similar companies is small. The difference between a successful startup and one that isn't can be tiny, but over time it grows and amplifies.

The key is this. Most things in startups, they can be done in a lot of different ways, and don't matter. But there are some critical factors where it makes a difference whether you have 0.01% failure rate or 0.00001% failure rate. It could be customer satisfaction. It could be allowing competition to take a tiny market share (see Microsoft, for example).

It's sometimes obvious what those factors are. Sometimes it's hard to identify them. But it's even harder to drive something toward 99.99999% success. A lot of people will be happy with 99.9%. It's not intuitive that there is 1,000X difference between the two. It seems unreasonable to a lot of people to go beyond 99.9% because it's hard for our brains to process the difference and get the emotional feeling of "wow, that's a big difference." It seems like nothing.

A further problem is that a successful startup is not designed to minimize mistakes. It's designed to maximize them. A startup culture doesn't lend itself to discipline. And to be frank, it shouldn't.

How do you reconcile the two?

One answer is the "unreasonable" founder. Some people seem to be wired to be a perfectionist. Some people seem to be wired to be unreasonable. Some people seem to be wired to ignore objections. Steve Jobs, for example, insisted that the insides of his computers be as beautiful as the outside. These people drive organization toward success because they see a huge difference between perfect and almost perfect.

The other answer is an operations person who's a perfectionist, who will bring discipline and provide the yin to the yang. Like Eric Schmidt at Google. Or have both a fanatical founder and an operations guy, like Steve Jobs and Tim Cook. It's a rare successful startup that doesn't have one or the other, because you have to be fanatical about at least some things if you're going to be successful.

Don't get me wrong - lean startup theory tells us you need to do a lot of imperfect things. I've written before about how failure is good. Those are all true. But we can't forget the importance of discipline. In most startups, there are a few things, perhaps 5%-10% of things, that really matter and deserve perfection.

For example, in our company, KindredBio, we aim for perfection in our clinical study design. Even a tiny difference in which dog we enroll in our trial, or how we evaluate the endpoints can mean the difference between success and failure. In a sea of creative innovators, we have disciplined operators. And people will tell you that while I'm laissez-faire about some things, I am fanatical about budgets and timelines.

The take-away is this: 80% of the things you do in a startup are very imprecise, and wild stab at things. But there are things that do matter. It could be customer service, for example. Or hiring creative people. Or scaling up quickly. For those things, it's not enough to be almost perfect. You have to obsess over it. You have to make it perfect.

“The reasonable man adapts himself to the world: the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man.”

― George Bernard Shaw