The Checklist Manifesto: Three Best Practices to Improve Venture Capital Performance

Summit day on Mount McKinley! Tent bound at 18,000 feet for two days in blizzard conditions, we were eager for our summit bid on the highest mountain in North America. When the howling winds finally relented, I geared up quickly keeping gloves on while donning crampons and roping up with temperatures at minus 30 degrees Fahrenheit.

After roping up, my partner and I went through a mental checklist as we inspected the other’s gear. Crampons firmly affixed on boots. Check. Climbing harness belt secured. Check. Large opposable carabiners locked in place. Check. Rope knotted correctly to harness. Check. The shared bond of a rope is the ultimate measure of trust. A fall by either partner puts both lives at risk. The bond of a rope affirms each partner’s shared commitment: you live and die together on the mountain. ??????

After three weeks together, such simple checks may seem mundane. Yet with three climbers already reported dead in the past week without a successful ascent on McKinley this season, we understood that nature calls the shots at high altitudes in Alaska. Minute mistakes carry grave consequences in extreme conditions. Only a few years earlier on a bid to be the first American woman to reach the summit of Everest, Marty Hoey fell 6000 feet to her death due to a rope that was incorrectly tied. She and her partner Jim Wickwire had not done the checks that we now routinely do before breaking camp.[1]

?

Checklists: Promoting Safety in the Airline, Construction and Healthcare Industries

No pilot today takes off without going through a checklist. After a deadly crash while testing a new airplane model in 1935 due to “pilot error”, Boeing devised a checklist for takeoff procedures to help pilots deal with increased speeds and complexity. The checklist worked surprisingly well and the new plane flew 1.8 million miles without further incident convincing the military to award Boeing a wartime contract for the B-17.[2] After the commercial airline industry adopted checklists, airline accidents per million miles declined by 95% in the 1960s.

The construction industry also adopted checklists in response to increased complexity. “Master Builders” oversaw construction for centuries relying on instinct and expertise acquired through apprenticeship. But by the middle of the twentieth century, general contractors emerged to manage increased specialization and relied on detailed checklists to manage subcontractors, construction schedules and requisition of building materials. Checklists have improved both efficiency and safety: buildings are constructed in a third less time while annual failure rates have dropped to just 1.5 per million high rise buildings according to a 2003 Ohio State University study.[3]

Rock star Van Halen devised a peculiar way to manage checklists. While on tour, his contracts specified that a bowl of M&Ms be provided backstage with all brown candy removed. When Van Halen found brown M&Ms in his bowl in Colorado, he peremptorily canceled the concert. His seeming whimsical aversion to brown M&Ms was instead a test. As later explained in Crazy from the Heat, Van Halen had huge productions requiring assembly of gear that filled nine eighteen-wheeler trucks. Ignoring the M&M clause in article 126 of the contract signaled that organizers may have overlooked other details in event preparation. In Colorado, the band found that local promoters had failed to observe weight requirements and the staging would have fallen through the arena floor.[4]?

Hospitals have adopted checklists only recently and found stunning improvements in safety and saved lives. This is most evident in intensive care units (ICUs) where urgent care is required as lives hang in the balance. Half of ICU patients experience life-threatening complications, yet an Israeli study found that ICU clinics make on average nearly two errors per patient per day.?

In 2001, Peter Provonost from Johns Hopkins Hospital focused on one common ICU challenge: central line infections, which had a ten-day line infection rate of 11%. Provonost leveraged support from Michigan insurance agencies to convince state hospitals to adopt a checklist involving five simple sanitary procedures, including washing hands, donning sterile apparel, and sanitizing a patient’s skin before and after insertion. The results published in the New England Journal of Medicine in 2006 were surprising. Adhering to these simple procedures in Michigan hospitals reduced infections by 60% saving an estimated $175 million and 1500 lives in eighteen months.[5]???

Checklists are now used widely across professions governing activities that range from mundane to mission critical. We organize our daily lives with to do lists. Master chefs consult recipes. Sports teams memorize playbooks. Drivers use directions and roadmaps. Teachers construct syllabi and lesson plans. Travelers assemble itineraries with matching packing lists. Event planners, contractors and architects use punch lists. Auditors and lawyers refer to due diligence checklists. Manufacturers construct checklists to manage production, quality assurance, and equipment maintenance. Retailers have inventory control and customer service checklists. Pharma uses checklists for clinical trials, regulatory compliance and laboratory procedures. Across all these use cases, checklists consistently reduce error rates and improve efficiency.

?

Checklists: Standard Practice in Private Equity but not Venture Capital

Despite widespread use in other industries, checklists are rarely used in venture capital. Geoff Smart conducted a study of 51 venture capitalists and found that only 12% evaluated potential investments using due diligence checklists. Yet VCs who used checklists had the highest success rate among the investors Smart surveyed delivering investment returns that were over two times higher than their peers. They also replaced just 10% of their startup CEOs compared with 50% of CEOs backed by other investors.

Yet Smart’s findings have not altered standard practice in the venture industry. After publishing a best-selling book Who discussing how to conduct management assessments, Smart reported his methods have influenced private equity and corporations but not the venture industry.[6]

Tom Alberg, a thoughtful investor and founder of highly regarded Madrona Ventures, offers a perspective on why most early-stage investors approach founder assessments as would an ‘Art Critic’ to use Smart’s terminology. Alberg observed, “Even with rigorous due diligence, most early-stage investments come down to an informed hunch.”[7] That many of today’s iconic tech firms – Amazon, AirBnB, Peloton, Pinterest, Salesforce and Uber to name just a few – had difficulty securing initial funding raises the uncomfortable question whether reliance on hunches is misplaced.

Charlie Munger, Warren Buffett’s partner at Berkshire Hathaway, acknowledged, “I came to the psychology of human misjudgment almost against my will; I rejected it until I realized that my attitude was costing me a lot of money.”[8] Munger developed a checklist of 100 or so mental models that he used to assess investments. Munger served as an able inverter for Buffett and used his list of mental models to evaluate investment risks for prospective Berkshire Hathaway investments. Using our earlier climbing analogy, Munger and Buffett worked well together for over fifty years, moderated each other’s tendencies and climbed high.?

Private equity has followed Munger and adopted checklists to manage due diligence processes. Standard practice applies well to private equity, which invests into established businesses in mature markets with a financial track record and referenceable customers. Venture startups often lack any of these traditional signposts.

Early-stage investing is still more art than science, yet Smart’s study suggests that checklists still apply in venture investing using different signposts. The venture industry is an efficient market over the long term and best practices will eventually prevail. If a systematic approach to investing consistently improves investment performance, then funding commitments should flow to those who adhere to industry best practice. ?

Several factors suggest that venture investors may do well to revisit their long-held practice of relying on hunches much as hospitals reluctantly reassessed their procedures two decades ago. Increased capital commitments to startups have raised the stakes of early stage investing. Deep tech investing and the application of technology across traditional industries have increased the variety and complexity of venture investing. The faster pace of investment decisions along with increased complexity and capital requirements may trigger a different set of investment practices much as they did in the airline and construction industries.

?

Checklists: Best Practices Applicable to any Investor or Entrepreneur

Checklists improve success rates in complex, unpredictable conditions as our examples in the airline, construction and healthcare industries illustrate. Checklists reinforce industry best practice and routinely improve adherence to basic procedures.

Yet many specialists resist checklists fearing that they erode the value of expertise and undermine discretion when exceptions arise. Like Munger and Master Builders, investors tend to reject simple aids to making decisions believing their profession is more art than science. Three best practices for checklists reinforce best practice and reduce biases that routinely skew investment decisions while freeing investors to focus on those situations where expert judgment may be best applied.

Keep Checklists Simple: When designing checklists for hospital procedures, Atul Guwande observed a tension between brevity and effectiveness. Longer lists specified detailed procedures but invite shortcutting and skipping steps. Trained professionals tended to ignore longer lists resisting micromanagement. Guwande found simple, short lists ultimately proved more effective as they served as useful reminders and promoted best practice while allowing specialist discretion where warranted.[9]?

At NGP Capital, we use checklists to help us assess founder market fit and product market fit, two common reference points for venture investors. Much as Guwande observed, we have found that short checklists such as those that assess on founder opportunity fit and product market fit are more effective than long, detailed investment checklists.

Do Confirm v. Read Do: ?Checklists may be administered in two ways. With Do-Confirm checklists, teams perform their jobs from memory and then review the checklist to ensure that all was done as expected. Read-Do checklists are followed step by step with tasks checked off as they go much as one follows a recipe.[10]

Appropriate procedure depends on the nature of the task, team experience and judgment required. Checklist construction differs depending on usage requirements. Read-Do checklists typically involve more detailed instruction, while Do-Confirm checklists are sparser but may be longer to serve as reminders for variations that might be otherwise be overlooked.

For experienced venture investors, a Do-Confirm checklist is more effective than a Read-Do checklist. A Do-Confirm checklist frees project teams to coordinate due diligence in a way that fits best with the investment timeline and startup’s specific situation while fulfilling all basic diligence requirements.

Monitors: In Thinking, Fast and Slow, Nobel laureate Daniel Kahneman distinguished between the many instinctual, daily decisions from the contemplative choices with outsized consequences that we make infrequently. For the latter ‘thinking slow’ decisions, Kahneman proposed including ‘decision observers’ to monitor decision making processes using standardized checklists.[11]

Designated observers ensure adherence to standard procedure and avoid natural shortcutting tendencies when checklists are self-administered. External observers were common in the above examples. Climbers inspect their partner’s gear. Copilots review checklists enabling the pilot to focus on flying the plane. General contractors monitor work of subcontractors at construction sites. Nurses or other observers monitor adherence to checklist procedures in a hospital. ?

At NGP Capital, we include an “Inverter” on all project teams when investments move into due diligence. The Inverter monitors the checklist ensuring the project team addresses key investment risks. An able Inverter offers a different perspective balancing natural optimism of the project team and serves as a wingman covering blind spots of the sponsoring partner. Like the relationship between Munger and Buffett, the Inverter must be respected by the project team to be effective. Partners with domain expertise from another region offer a distanced, respected perspective to the process.

?

Concluding Thoughts

Process sets us free. The natural tendency is to consider process as a constraint. Yet well-designed processes improve efficiency while reinforcing best practice and reducing errors and oversights. Investments are one way, ‘thinking slow’ decisions that deserve careful consideration according to the ‘prudent man’ investment principles. Due diligence checklists can ably facilitate a streamlined, yet rigorous investment process. When checklists adhere to the three best practices described above, they free investors to use their expert judgment where it is best applied.

May you climb high and reach all your summits! ?

?

[1] Jim Wickwire and Dorothy Bullitt (1998). Addicted to Danger. Pocket Books. Page 167.

[2] Guwande, A. (2009). Pages 32-34.

[3] Guwande (2009). Pages 49-71.

[4] Guwande, A. (2009). The Checklist Manifesto. Page 80.

[5] Guwande (2009). Pages 37-44.

[6] Guwande, A. (2009). Page 172.

[7] Alberg (2021). Flywheels.

[8] Bevelin(2018). Seeking Wisdom.

[9] Guwande (2009).

[10] Guwande, A. (2009). Page 123.

[11] Kahneman, D. (2011) Thinking, Fast and Slow discusses different thinking processes. Noise: A Flaw in Human Judgment (2021) proposes decision observers and checklists.