How to increase your odds of success when crowdsourcing innovation ideas?

Over the last few years, together with a wonderful team with whom I learned a lot (Stefan Stremersch, P.K. Kannan and Hyoryung Nam), we have conducted a series of empirical studies in order to try to shed some light on the question in this title. I believe the answers we obtained bring a valuable fresh perspective for firms willing to increase the odds of discovering and developing high quality and high impact innovation ideas through crowdsourcing initiatives, namely innovation tournaments.

I am very happy that the results of our work - which involved countless hours of research, real-life observation of the phenomenon, and several empirical studies - will soon be disseminated through our forthcoming publication in the Journal of Marketing (March 2019). In this short LinkedIn article, my goals are to (1) offer a brief snapshot of our findings (technical details are all in the paper), and (2) offer a glimpse of the personal journey that I went through while working on this particular research project.

The setting: Crowdsourcing innovation ideas through innovation tournaments

Firms increasingly use innovation tournaments to crowdsource innovation ideas from customers (or employees). For instance, in 2012 PepsiCo’s ‘Do Us a Flavor’ tournament helped the company create the ‘Cheesy Garlic Bread’ flavor which contributed to an 8% increase in sales in the three months following the tournament. Other successful new products generated through innovation tournaments include a line of rugged Dell laptops for marine use and thematic Lego sets (e.g., Back to the Future’s DeLorean and Ghostbusters’ Ectomobile).

In an innovation tournament, a firm launches an open call to ask a large crowd of ideators to submit innovation ideas into an online idea generation platform. After a prescribed period of time, the firm selects at least one winning idea from those submitted. To gauge the success of these innovation tournaments, firms routinely monitor the number of ideas and the number of participants in a tournament. Yet, they do not always consistently monitor or stimulate ideators' engagement. Should they? How?

The Studies

To examine these questions, we conducted several empirical studies in the last few years. Specifically, we conducted two longitudinal experiments using a commercial innovation tournament platform (Cognistreamer).

Figure 1. Snapshot of the innovation platform we used in our studies (Cognistreamer)

In each of these experiments, we organized a tournament called “ESE Innovation Tournament” where we invited students of the Erasmus School of Economics, the Netherlands (at the occasion of its centennial celebration), to contribute ideas that would have an impact on the school by 2030. Over several rounds, we then experimentally manipulated the type of moderator feedback given to each ideator over time to measure the impact of feedback type and timing on ideators’ engagement, or participation intensity (i.e., how frequently ideators view their ideas and update their ideas over the course of the tournament). 

Figure 2. Example of longitudinal experiment design

We also ran a large survey among innovation executives at 1,519 firms (out of which 516 - or 33.95% - had already run an innovation tournament on an online platform), to examine whether higher participation intensity leads to better ideas, over and above the number of participants and number of ideas submitted to a tournament.

Brief snapshot of our findings

The results were unequivocal:

• Participation intensity is a critical driver of idea quality in innovation tournaments. Its effect was at least as important as the effect of number of ideas and number of ideators. 
• To stimulate ideators’ participation intensity, firms should carefully design their moderator feedback strategy (i.e., type and timing of feedback).  Specifically, we show that negative feedback trumps positive feedback (note: by negative feedback we mean "constructive criticism" that highlights work to be done and should not be unnecessarily detrimental), if provided early (rather than late) in an innovation tournament. We also find that "sandwiching" negative feedback with positive feedback (i.e., feedback that praises the work accomplished so far) is not more effective than negative feedback in isolation.

Implications

We suggest that participation intensity should become a behavior to monitor, a metric to report and an outcome to incentivize.

Moreover, our findings about the effect of moderator feedback on ideators' engagement in innovation tournaments help firms actively stimulate participation intensity. Specifically, organizers of innovation tournaments should train moderators to offer feedback that challenge participants’ ideas and highlight to participants the “work that still needs to be done” for the idea to be successful and to do so early rather than late in an innovation tournament. 

My personal journey: A short reflection

It is hard to express how fortunate I am for the energy, learning and growth opportunities that working in this project opened up. The project was large, ambitious and challenging. Designing and implementing an innovation tournament, with a real marketing campaign (open call), and where moderator feedback was carefully manipulated entailed many challenges, which meant there was never a dull moment in this project. I extract three key learnings from this project.

First, team diversity really is a blessing. Our research team included four different nationalities. Such diversity was, in my opinion, one of the team's key assets. Collaboration was always rich and inspiring. We frequently had divergent views. In my opinion, it was precisely such divergent perspectives that made the collaboration so rich and our profiles really complementary.

Second, interacting with managers to gather a view "from the trenches" is inspiring and enriching. While working on this project, besides gathering feedback from academic colleagues (e.g., in conferences, at our department...), we also had the opportunity to discuss these questions with several managers who were designing and managing their own innovation tournaments. Being able to look at the same questions we were researching academically through the eyes of managers was very enriching to our project and to me personally. On the one hand, by pushing us to seek answers focused on business impact, executives with whom we interacted helped us stay focused on answers that truly matter for firms. On the other hand, by incorporating executives' views and feedback in our research design, we gained valuable knowledge and insights that we can use to give back to the managerial community, in the form of more informed and evidence-based answers to questions they may still have. It is a win-win partnership, in my opinion.

Third, despite all the technological advances in connectivity, face-to-face collaboration still matters. The geographical spread added an interesting twist to the collaboration. In the first half of the project, Hyoryung was still in Rotterdam before moving to Seattle. As of that point, we were frequently working from at least three different locations across five countries: Rotterdam (Netherlands), Maryland & Seattle (USA), sometimes Barcelona (Spain), sometimes Stekene (Belgium) and sometimes Porto (Portugal). Who would have imagined this type of collaborations just a few years back? Still, in this project I felt that face-to-face meetings were still very important moments. Foreseeing this, we ensured we had several opportunities per year for face-to-face meetings. An afternoon or day reserved to work on the project before, during or after major conferences we all visit, a very productive trip that Stefan and I made to Maryland where we were wonderfully received by P.K., etc. When we needed to creatively solve challenging problems together, such face-to-face moments proved crucial in unlocking major progress.

All in all, I hope readers are as excited as I am about the topic and that the article stimulates further debate and research on these topics.