Open innovation outcomes, by the numbers

Many executives, when tasked with innovation, think they have an idea problem. But as any participant in a sticky-note brainstorm session knows, the real problem isn’t coming up with ideas, it’s closing the gap between concept and making it real. One of the central tenants of open innovation challenges is that they can accelerate the identification and strengthening of solutions in a short amount of time.

We recently shared what we’ve learned about open innovation challenge design over the past seven years. Anecdotally, we’ve heard success stories from winners and finalists of challenges we’ve designed and produced, and they’ve shared how these challenges have influenced their trajectories. It prompted us to ask “where are they now?” and “what can we learn from their experiences?”

We surveyed 60 prize recipients (including semi-finalists, finalists, and winners) of 14 large-scale, multistage challenges we ran on behalf of government, nonprofit, or private sector clients. All of these challenges included a virtual accelerator component, where selected teams participated in learning and prototyping modules designed to help them refine and improve their solutions.

Spotting potential

The basic premise of open innovation is that novel solutions to important problems can come from anyone and anywhere. Furthermore, our survey shows that winning solutions enter challenges at different levels of maturity. Some prize recipients had only an early concept when they entered a challenge; others had developed existing solutions exploring a new area or looking to refine their offering.

For most challenges, we look to cast a wide net — remaining open to everything from concepts to richer solutions from known solvers as well as innovators in analogous spaces — and favor judging rubrics that screen for potential instead of simply evaluating the existing state of the solution.

Open innovation is both a short game and a long game. While selected teams make rapid progress during the challenge, what happens over the subsequent months and years tells the full story. Almost all (92%) of teams surveyed continued developing their solutions after the challenge. The validation that comes along with being selected as a prize recipient can provide momentum for ongoing development.

Everyone loves an underdog, and we’ve seen underdogs become market-ready in a relatively short time. Out of 13 winning teams, only three had market-ready solutions when entering the challenge. Today, 10 of 13 are market-ready and two are working prototypes. In a multistage open innovation challenge, a concept can compete against a market-ready solution.

Market indicators

There are a number of proxies for measuring progress, and one is the level of interest from the investment community. We know that raising money does not guarantee success; venture-backed companies regularly flop. But we also know that capital is critical to ongoing development, especially for prototypes addressing the thorniest problems. We found that two-thirds of teams surveyed raised funding after the challenge — in the form of grants, venture capital, seed or angel investments, crowdfunding, and prizes — in amounts ranging from $5,000 to $25 million. We conservatively estimate that prize recipients have gone on to cumulatively raise $100 million, suggesting that investors see challenge prizes as a positive indicator.

We’ve seen just how attractive these solutions are to early stage venture and strategic investors. A few examples of post-challenge fundraising that caught our attention:

• Osso VR raised $2 million in venture capital after winning the EdSim Challenge.
• Smart Sparrow, a finalist in the EdSim Challenge, recently received a $7.5 million investment from global education nonprofit ACT.
• IST Research, a finalist in the Rethink Supply Chains Challenge, won $4 million in government contracts. Another finalist, Good World Solutions, received $1 million in grants.
• Wellframe, a finalist in the 2011 Data Design Diabetes Challenge, raised $25 million in venture capital and went on to win the Merck Heritage Provider Network Innovation Challenge.
• Ginger.io has raised more than $25 million in Series A and Series B rounds since winning the 2011 Data Design Diabetes Challenge and going on to win the Alzheimer’s Challenge.
• LiveHealth, a finalist in the 2012 Data Design Diabetes Challenge, received over $5 million in grants.
• After winning the Janssen Connected Care Challenge, RightCare Solutions’ Discharge Decision Support System raised more than $10 million in venture capital and $3 million in grants. They told us the cash prize served as seed funding to finalize their prototype, line up a pilot customer, and secure Series A funding. The company was ultimately successfully acquired by naviHealth.

A second interesting indicator of potential value creation is the filing of patents — and eight teams reported filing at least one patent related to their solutions since participating in a challenge. Two of those teams — Diabetty and Sugarpod by Wellpepper — filed patents for their voice technology solutions after participating in the Alexa Diabetes Challenge last year.

Because we’re looking at challenges over the past seven years, we’ve seen solutions evolve in different ways. For example, the Purdue University team that was named co-winner of the 2014 FDA Food Safety Challenge told us they have developed more advanced prototypes in partnership with other investigators and the challenge co-winner; they continue to work with scientists at FDA as they create an instrument for field testing. In addition to partnerships and collaborations, some of the biggest post-challenge bets have been acquisition: 5% of teams surveyed sold their solutions after the challenge, and 28% folded their solution into another product or service.

Fully one-quarter of prize recipients surveyed were not even a company when they entered the challenge as a team of individuals. However, following the challenge, nearly half of those teams went on to form companies.

Wants vs. needs

How do you use an open innovation challenge to attract promising solutions and accelerate development to achieve these outcomes? Challenge design is critical — and should recognize that what solvers think they want isn’t always what they need. Balancing incentives, structure, and learning helps to push the best solutions forward.

The right incentives can attract different types of solvers to a challenge. Prize money is obviously a powerful incentive for entering a challenge, but it’s not the only thing that drives quality submissions. In fact, a team’s motivation for entering can depend on the maturity of their solution before the challenge, and money matters more when a solution is more mature. Cash awards were the primary motivation to enter for half of teams with market-ready solutions. But among teams entering with only a concept or idea at the time of entry, half said their primary motivation was an interest in exploring the problem space.

Not surprisingly, challenges with larger cash prize purses ($250,000 and higher) saw a larger percentage of teams entering because they were motivated by a monetary prize. For challenges offering a cumulative prize purse of $250,000 and higher, 30% of prize recipients surveyed said cash awards were their primary motivation for entering. Conversely, for challenges with smaller cash prize purses (less than $250,000), only 10% of teams surveyed said the monetary prize was their primary motivation for entering; they were more interested in exploring the problem space (35%), earning validation (25%), or gaining exposure to people and organizations involved in the challenge (15%).

Overall, only 10% of teams surveyed said learning opportunities — in the form of a virtual accelerator that could include a boot camp, piloting, and/or mentorship — were the primary motivation for entering. But after the challenge, nearly half of teams surveyed (47%) named learning opportunities as an important benefit of participating, and one-third (33%) said they valued connecting with challenge sponsors, partners, mentors, judges, or other participants.

The virtual accelerator component of a multistage challenge helps solvers connect with mentors and end users who can provide much-needed feedback, as well as learning modules to deepen understanding of relevant skills and topics. Empathy exercises tend to be particularly enlightening; we’ve seen teams pivot based on what they learn from interactions with “real people” such as patients, students, and city managers. A better understanding of end users and real market forces can help finalists develop truly innovative solutions that benefit humans.

We asked prize recipients to share — in their own words — how participating in a challenge impacted the development of their solutions. Here’s what they had to say:

We’ll be examining how companies approach open innovation and how the practice may evolve in the coming years. Stay tuned for more stories from prize recipients, as well as findings from our survey of organizations that use open innovation. (If you manage open innovation programs and would like to participate in the study, send an email to [email protected].)

Photo by Teddy Kelley on Unsplash

Originally published in the Lab Report on April 17, 2018 by Janna Gilbert, Sara Holoubek, and Elizabeth Bowling . The Lab Report helps executives make sense of emerging trends and new opportunities. Sign up for the newsletter, and we’ll send insights straight to your inbox.