Assessing business incubation: A review on benchmarking

This paper studies benchmarking on Business Incubators (BIs), aggregating the state of knowledge about benchmarks for BI assessment. Based on 136 papers that evaluate BI performance, this study first investigates all Key Performance Indicators(KPIs) provided by 42 BI assessment studies and the relevant benchmarks for each KPI. The final benchmark list is created by taking seven of the most prominent benchmarking studies as a basis. With this framework, this research contributes to the literature on BI assessment by providing comprehensive and aggregated benchmarks for these important innovation habitats. A final list of 33 benchmarks is provided so that BI assessment, for which no common framework exists, becomes easier. This study suggests that with the difficulty of creating a common assessment framework for BIs, it is more reasonable to create a common benchmarking framework that accounts for regional and industrial differences. The results could be subject to a multiple criteria decision analysis. Our paper is the first to aggregate the diverse information on benchmarking and provide researchers and BI managers with a base list of benchmarks.

(Originally published on Elsevier)

1. Introduction

Business incubators (BI) date back to the 1950s (Mian et al., 2016). After the first organization was established, the number and types of BIs increased rapidly (Bruneel, Ratinho, Clarysse, & Groen, 2012; Campbell & Allen, 1987; Cornelius & Bhabra-Remedios, 2003). By 2012, the number of incubators reached seven thousand globally, according to Knopp (2012). It can be said that this rapid increase is because economies are nurtured by the growth of incubation programs and because BIs are stimulated by industry needs (Abetti, 2004).

BIs are usually related to technology transfer offices and technology or science parks due to their role in supporting technology and accelerating the success of new technology-based firms (Abetti, 2004; Hackett & Dilts, 2004; Phan, Siegel, & Wright, 2005). Two concepts, “incubatee” and “business incubator”, will be widely used throughout this study. Incubatees are the tenant companies in BIs. The definition of BI is controversial (Dee, Livesey, Gill, & Minshall, 2011). Hackett and Dilts (2004)define a BI as a “shared office-space facility that seeks to provide its incubatees with a strategic, value adding intervention system of monitoring and business assistance”.

Another broadly used definition is given by Lewis, Harper-Anderson, and Molnar (2011) in which a BI is defined as

A multi-tenant facility with on-site management that directs acceleration of the successful development of entrepreneurial companies through an array of business support resources and services, developed or orchestrated by incubator management, and offered both in the incubator and through its network of contacts.

According to Dee et al. (2011), there are two main reasons for incubators to operate. The first one states that BIs address market failure (OECD, 1997; Phan et al., 2005), and the second one argues that incubators accelerate the entrepreneurial process and offer support for new ventures (Hansen, Chesbrough, Sull, & Nohria, 2000). However, generally, the main goal of incubators can be summarized as producing successful firms that will graduate to be financially viable and freestanding (Aernoudt, 2004; Tornatzky, Sherman, & Adkins, 2001). In other words, we can summarize the roles of the incubators for the community as follows: to foster job creation (Knopp, 2012; Wynarczyk & Raine, 2005), to develop new ventures (Allen & Rahman, 1985; Amezcua, 2010b; Scillitoe & Chakrabarti, 2010), to support the entrepreneurial community (Bruneel et al., 2012) and to promote economic development (CSES, 2002; Lalkaka & Abetti, 1999; OECD, 1997; Phan et al., 2005; Schwartz & G?thner, 2009; ?ehito?lu & ?zdemir, 2013). Hence, one can state that the main rationale of business incubation appears to be contribution to economic development either regionally or nationally (Mian, 1996).

These potential remarkable effects of BIs on economic development generate the need for an evaluation of their performance (Phan et al., 2005). The effort of incubator managers to demonstrate their success in order to make their funding sustainable can be considered one base of this justification; however, their financial dependency may result in some biases (Hackett & Dilts, 2004). This is why associations and governmental organizations frequently evaluate the impact and performance of BIs (Dee et al., 2011). Despite the fact that there is no common method for assessing incubators and there are many challenges for evaluating BI performance (Hackett & Dilts, 2007; Schwartz & G?thner, 2009), it is still important to estimate incubation impact for the community, incubators and incubatees (Hackett & Dilts, 2004).

Although there are many studies about BI performance evaluation, there is a gap in the creation of a benchmarking framework for BI managers and policy makers so that they can easily compare BI programs with others. In this paper, the benchmarking issue is studied for incubator managers to aggregate information about key performance indicators and preferred benchmarks for the sake of making incubator assessment easier.

This paper is structured as follows: In the next section, we study the literature on benchmarking, considering that there are a significant number of studies suggesting KPIs but that few provide the benchmarks for them. In Section 3, we focus on the method used to reach the relevant data and studies. In Section 4, the results of the data analysis are shown. Finally, in Section 5, the conclusions reached in this study are exposed, and a discussion on the benchmarks for BIs is presented.

2. Literature review

2.1. Historical context

The first place that established an incubation process was Stanford University in 1951; it was named Stanford Industrial Park and then later renamed Stanford Research Park. It is considered as the first Incubator. It was followed by Industrial Center of Batavia in New York, in 1959. These two institutions pioneered the business incubation industry (Mian et al., 2016). In 1960, the BI movement reached Europe with the first BI established in United Kingdom, and in the 1970s, several BIs started to assist companies all over Europe. Two important steps accelerated the spread of the BI concept around the world: the first was linking.

BI to university research in the 1980s (OECD, 1997), and the other is the global expansion of high-quality Internet (Grimaldi & Grandi, 2005). The rise of internet-related ventures, quick access to markets and capital and the development of entrepreneurial activities catalysed BI growth (Autio & Parhankangas, 1998; Chinsomboon, 2000). After the 90s, a new BI model was created: virtual incubation. This new model did not require shared offices or resources and was focused on information, communications and technologybased companies. Moreover, the BI industry met another new model in 2003, the business accelerators (BA). The first BA was Y Combinator, established in Mountain View, California. BAs differ from BIs by providing seed funds, being cohort based, focusing on short term processes and having for-profit structures (Miller & Dalziel, 2018; Van Huijgevoort & Ritzen, 2012).

The literature notes that the evolution of BIs mostly occurs in three phases, as argued by Mian et al. (2016) and Bruneel et al. (2012). Considering these two coinciding studies, the classification of the BI industry used in this paper is shown in Table 1.

2.2. Challenges of business incubator evaluation

There is a variety of understanding regarding what is considered success for BIs, as well as a common set of related KPIs for the business incubation phenomena (Vanderstraeten & Matthyssens, 2010). For example, Hackett and Dilts (2007) defend that the success of an incubation performance is related to incubatee growth. However, Autio and Parhankangas (1998) and Bergek and Norrman (2008) argued that incubator success is related to the degree to which the support arrangement is able to meet the objectives of the incubatee.

Additionally, the difference between success and failure is not clear for different situations. For example, Avnimelech, Schwartz, and Bar-El (2007) give only two degrees of success, high and low, whereas Hackett and Dilts (2007) provide a five-scale mechanism in the success-failure spectrum.

Furthermore, there are also different approaches to incubation evaluation that concern “evaluation focus”. For instance, Costa-David, Malan, and Lalkaka (2002), Lalkaka (1996), Bergek and Norrman (2008) and Adkins (2001) argue that we need to focus on the incubation process for a successful performance evaluation, whereas Sherman and Chappell (1998) discuss that the focal point should be on incubation process rather than on outcome and impact. Moreover, the OECD (1997) states that the success of incubators depends on meeting the objectives of its stakeholders. In other examples, Banner and Gagne (1995); Mian (1997) and Robey (1986) discuss common approaches for assessment, whereas Vanderstraeten and Matthyssens (2010) expose obstacles for each of them.

Finally, although there are a significant number of studies providing performance measures/KPIs for BI performance assessment, few studies present benchmarks for these measures. Moreover, the provided benchmarks and KPIs differ for each study.

This contradiction in the BI performance assessment literature makes performance evaluation difficult for managers and institutions. Thus, this fact makes the creation of a benchmark framework necessary to give useful directions to manage incubators.

2.3. BI assessment approaches and benchmarking

In general, there are three research approaches used in the assessment of the performance of BIs (Dee et al., 2011): The control group concept adopted by Amezcua (2010a); Dettwiler, Lindel?f, L?fsten, and Apr (2006); Lindel?f and L?fsten (2002) and Siegel, Westhead, and Wright (2003), the benchmarking concept supported by Bhatli (2015); CSES (2002); Dichter and Marchand (2017); Knopp (2012); Miller and Dalziel (2018) and UKBI (2009) and the in situ concept assessments suggested by Bergek and Norrman (2008); Grimaldi and Grandi (2005) and Patton, Warren, Bream, and Feb (2009). Each of these approaches has several advantages as well as challenging obstacles. One challenging difficulty for the control group approach, for example, is collecting data from non-incubated firms (Amezcua, 2010a). In addition, this kind of study can have strong restrictions to be generalized. The in situ approach also has a significant complexity of evaluation (Schwartz & G?thner, 2009), such as regional differences (Bergek & Norrman, 2008), objective differences (Bigliardi, Dormio, Nosella, & Petroni, 2006; Cornelius & Bhabra-Remedios, 2003), and different types of incubators Bigliardi et al. (2006) and data availability Schwartz and G?thner (2009).

Benchmarking was defined by Sourching (1997) as “a systematic approach to business improvement where best practice is sought and implemented to improve a process beyond the benchmark performance”, and it is used in many assessment-related pieces of literature (Hinton, Francis, & Holloway, 2000). Although benchmarking inherits similar difficulties to in situ research, as it is one of the most adopted methods for evaluating performance (CSES, 2002; Dee et al., 2011), there are indeed a significant number of benchmarking studies Caiazza (2013); Costa-David et al. (2002); CSES (2002); ECA (2014); Lewis et al. (2011); UNECE (2001). According to Miller and Dalziel (2018) benchmarking BIs helps them to better understand best practices as well as it makes BI manager to think differently as a result of exposure to new models and processes and to engage with organisational learning. It can be said that benchmarking is challenging (Allen & Rahman, 1985) but doable (Bearse, 1998; Hackett & Dilts, 2004; Miller & Dalziel, 2018) and can be a good tool for assessing BIs.

3. Method

In this study, a three-level measure set, which is provided by Dee et al. (2011) and Hamdani (2006), was used in order to create a comprehensive list of successful indicators and their related benchmarks, as listed: Community level, which involves mostly impact studies; Incubation level, which involves elaboration on incubation process; and Incubatee level, which focuses on incubatees. More specifically, a measure is considered at community level if it is ready to contribute to the region. Incubator-level measures are related to the incubator facility, process and network. Finally, incubatee-level measures are related to incubatee success but not to the readiness to impact regional development.

In this study, as detailed below, 352 studies were surveyed, and considering their content, data and originality, 214 were then eliminated considering their scope, impact and originality. The remaining 138 studies form the basis of our study. To search for the papers, the academic databases Science Direct, Web of Science and Scopuswere used. To target BI performance evaluation literature, the following terms were used:

“Business + incubation”, “technology + incubator”, “science + parks”, “university + technology + transfer + office”, “business + incubation + benchmarking”, “success+of+business + incubation”, “business + incubation + assessment”, “impact+of+business + incubation”, “KPIs + for + business + incubation”, “performance + evaluation+of+business + incubation”, “business + incubation + best + practices” and “performance + measures + for + business + incubation”. Additionally, new papers were identified by using the bibliographies of previously searched studies that were found as described above. All papers published from 1985 to June 2017 were considered. For this work, peer-reviewed journal papers as well as books, special reports, technical reports and theses were examined.

While 42 studied papers provided 65 KPIs, 33 of which are common for at least two papers, seven institutions introduce 61 different benchmarks. Basic calculations were made on some of the existing benchmarks to get this number to 61. Even though a study does not provide a benchmark independently, if a basic calculation results in that measure, the result is considered as a KPI for that study. For example, two of the reports, CSES (2002) and OI (2006), provide average jobs per tenants as a KPI directly, while another two, Dichter and Marchand (2017) and Knopp (2012), separately provide just the number of tenants and total of jobs created. The number of jobs created was divided by the number of tenants for subsequent institutions to calculate the same KPI (average jobs per tenants). In addition, in the CSES (2002)study, the provided averages were considered as benchmarks for the relevant KPIs for which the authors did not provide a benchmark because of some restriction issues. Moreover, since the distribution of firm performance measures is not normal in most cases (Bearse, 1998), it is preferable to use the median of the measures provided in the six benchmarking studies for creating the final set of benchmarks.

In addition, not every benchmark refers to all KPIs since each study creates a different type of assessment framework and KPI. For example, one study may consider the total number of employees as a KPI, while the other includes employment growth or jobs created in its framework. In this case, both were considered as the same KPI when calculating the number of KPIs in the literature.

Most of the studies that include performance measures were found regarding impact, benchmarking and best practices, and some of them also appeared in literature reviews. While searching the provided KPIs for business incubation, it can be noticed that there are a significant number of studies establishing KPIs, but only a few papers provide the benchmarks for those KPIs. Moreover, some studies discuss the outcome of best practices as benchmarks of KPIs. Therefore, the number of papers that provide benchmarks and that can be a reference study for benchmarking literature is even smaller, as seen in Fig. 1.

Fig. 1. Chart of BI performance assessment studies.

After researching the literature, fourteen basic studies can be appointed in providing benchmarks for KPIs. Seven studies were excluded because either they are some combination of existing benchmark studies or they only provide KPIs from some best practices. The final set of papers used as a basis for this study can be seen in Table 2.

As shown in Table 2, all seven studies are institutional reports, which can be explained by the fact that the creation of benchmarks is usually done by official agencies, as doing so requires an authorized background.

4. Data and results

After following the method and identifying relevant references we now analyse the institutional reports and the 42 papers that provide KPIs.

The Center for Strategy & Evaluation Services (CSES, 2002) carried out a study with 77 incubators from EU member states for the European Commission. CSES then implemented a 5-step benchmarking methodology and came up with 16 KPIs. A group of managers from 11 different countries supervised the study. In the study, researchers created a benchmarking framework and then tested it with interviews and a survey (CSES, 2002). The framework consists of 5 steps: The model step, which is the creation of a basic model for BI including basic BI functions, setting up requirements based on the existing literature, and deriving many key best practice issues that help define benchmarking indicators; The second step is named best practice issues; The third step consists of performance drivers in which key performance drivers are highlighted from the model; The fourth step is about data, and researchers carried out a survey of both incubators and their clients; The final step – best practice guidance – offers key actions that should be taken to reach a best practice level (CSES, 2002). One of the significant key conclusions of this research is the need to focus on the results, and hence impact, of the BIs for performance evaluation. However, the three categories of suggested benchmarks do not contain that many impact benchmarks. These three categories of benchmarks are setting up and operating, incubator functions and evaluating services and impact (CSES, 2002).

Differing from CSES, Oxford Innovation (OI, 2006) developed a questionnaire for benchmarking BIs instead of creating a framework. For this, OI (2006) conducted a study with nine European incubators for the Gate2Growth Incubator Forum (GGIF), which is also supported by the European Commission (OI, 2006). Studies also compared their benchmarks with the CSES study. Since this brings some new KPIs different from the ones on the CSES framework, they were included in this study. The primary motivation of OI's study is to provide a questionnaire for GGIF that creates an online benchmarking tool. They also executed a pilot benchmarking study with 9 GGIF member incubators using their survey.

Another comprehensive study is the one conducted by InBIA (Knopp, 2012). InBIA has been considered one of the most important institutions in the incubator industry and considerable research refers to its findings, as well as to its definitions. InBIA has been surveying BIs throughout North America to keep track of industry trends since 1989. Previous benchmarking papers, such as Qian, Haynes, and Riggle (2010) and Amezcua, Grimes, Bradley, Wiklund, and feb (2013), considered the 2007 or 2009 versions of InBIA's state of knowledge study. In this paper, we included the newest one that was conducted with 235 incubators in the USA in 2012. The report neither evaluates the BI impacts nor tries to determine best practices. It simply surveys randomly chosen BIs to understand the state of BI industry. Since the author provides medians as the base numbers of success for a BI, the data of the report is included in our final benchmark list.

ECA (2014) is the study from European Court of Auditor for assessing BIs financed by the European Regional Development Fund. The researchers tried to understand whether the funded BIs had successfully supported new companies. Hence, this is an impact study. The report provides benchmarks by studying BIs in four of the EU-member states (the Czech Republic, Spain, Poland and the United Kingdom). They created their benchmarking framework based on the literature about benchmarking BIs.

One recent study for benchmarking is UBI-Global's report, which provides 24 KPIs after intensive work with 340 incubators from 64 countries (Bhatli, 2015). The study, in fact, is a ranking analysis providing best practice evaluations from which benchmarks can be derived. The study can be categorized as a combination of a best practice study and an impact study as it employs a ranking index. The study evaluates the BI impacts, as well. Their research framework is composed of three components: evaluation of the value for the ecosystem, value for clients and BI attractiveness (Bhatli, 2015). This framework is somewhat parallel with the CSES's three categories of setting up, incubator services and community impact. The main difference in UBI-Global's study is that it is the only private institution report, because the others are either public institutions or supported by a public entity. Another difference for this study is the scope of the analysed BIs. Previous reports studied BIs mostly from a regional (Europe or USA) perspective, however UBI-Global conducts a study concerning BIs from all over the world.

The other report which provides benchmarks is written by The Evidence Network Inc. (TEN) located in Waterloo, Canada. The Evidence Network Inc. (TEN) has been evaluating different types of business support programs in BIs since 2009. The Evidence Network (2016) conducted a study surveying 43 incubators and accelerators from Finland, China and Canada. The research question of the report is how policy makers can support innovation and economic development through business incubation. The authors studied this question by analyzing the performance of BIs and the factors that explain the impact of BIs. So, this study can be regarded as a combination of an impact and a benchmarking study. The report provides 10 different benchmarks after a multivariate regression analysis of the mechanisms of impact.

The most recent study giving benchmarks for BI assessment is from the European Business Innovation Network (EBN). It surveyed 132 (2014), 120 (2015) and 120 (2016) member-BIs. From the report, 23 KPIs for a “typical business incubator” can be derived. EBN uses an online self-assessment tool for collecting data from its members. The report is the result of a three-year data collection from 2014 to 2016. Generally, EBN's benchmarking framework is composed of two parts: networking and data collection (Dichter & Marchand, 2017). The second part was explained above. The networking part of the framework is the fixed technical event in which the best practices from their network are exchanged. This also shows us that the benchmarking approach of EBN is also based on best practice issues.

Lastly, a final benchmark list was created (See Table 3) by using the information in the seven main studies with KPIs that are common to at least two studies in the literature (See Fig. 2).

Note: *These three KPIs do not have any reference benchmark in six benchmarking studies. However, since they appear among the most used KPIs in the literature, we include them in the final set of benchmarks and consider the information in the relevant study as their benchmarks. **Since the newer version of the reference report does not provide a benchmark for this KPI, we considered the information provided in the previous version of the paper.

Fig. 2. Framework: Creating the final benchmark list.

The analysis of 42 papers showed that the number of assessment papers with KPIs is still increasing, which implies that the BI assessment literature is still developing (See Fig. 3).

Fig. 3. The distribution of studied KPI giving papers to decades.

Using this benchmarking framework, shown in Fig. 2, a final 33 benchmarks for BI assessment were used, as can be seen in Table 3. The final benchmark list contains 4 incubatee-level KPIs, 20 incubator-level KPIs and 9 community-level KPIs (Fig. 4). Moreover, most of the KPIs in the total set of KPIs are also incubator-level. Separately, each of the 7 reports mostly contain incubator-level KPIs. This implies that the literature on BI benchmarking focuses mainly on the incubation process for evaluating a BI's performance. Furthermore, three recent studies, Bhatli (2015); Dichter and Marchand (2017) and ECA (2014), suggest most of the community- and incubatee-level KPIs (See Fig. 4). This implies that the community-level KPIs were only recently considered.

Fig. 4. Distribution of benchmarks according to organisational level in the seven selected studies (%).

EBN contributed to a final set of benchmarks with its 18 benchmarks. OI contributed with 12, CSES with 13, InBIA with 11, UBI with 8, ECA with 7, and finally TEN with 5 benchmarks.

5. Conclusion and future implications

The increasing number of BIs and related studies show that BIs are good catalysers for economic development. Although there is a shortage of common assessment frameworks, management strategies and even definitions, there are a significant number of studies on BIs. The three most common subjects in the BI literature are impact, management and assessment-related issues (Mian et al., 2016). This implies that performance evaluation is the most pressing issue in the industry, of which benchmarking is a subset. In this study, the literature on benchmarking BIs was reviewed, and the six most comprehensive benchmarking studies were explored. The results were found with some significant implications.

First, as many studies state, there is neither a common framework for performance evaluation nor a common set of benchmarks due to several constraints such as the variety of incubator types, goals, regions and stakeholders. Hence, all suggested sets of benchmarks can be applicable for a set of constraints or for a definite region. However, since some common aspects of many incubators were presented, creating a benchmarking framework as well as an assessment framework can help managers to compare their outcomes or processes with others.

Second, there is noticeable interest in BI assessment and benchmarking, but few studies provide real benchmarks. In most studies, there was an endless list of KPIs but not the benchmarks for them. This may stem from the difficulty of creating common indicators.

Third, although some research cited that there was a tendency to focus on community- and tenant-related indicators for performance evaluation at the beginnings of 2000s (Lofsten & Lindelof, 2001; Mian, 1997; O'Neal, 2005; Voisey, Gornall, Jones, Thomas, & Jul, 2006), incubation process-related KPIs are still commonly used today. On the other hand, the KPIs in more recent studies from UBI-Global, ECA and EBN appear to be related to the community effect of tenant firms. This may indicate that the community effect of BIs is recently starting to draw interest in the industry. This requires future studies on BI assessment to consider more stakeholders such as public decision makers, universities, governmental bodies, investors, unincubated startups, and graduated companies in the community. Additionally, because the community-related indicators are also limited to specific regions, future studies may consider a classification of the development level of the subject region. Separate frameworks for developed, developing and underdeveloped regions will produce more feasible outcomes for community-level BI assessment.

Finally, even though the business incubation industry dates back to the 1950s, its tangible impacts on communities have recently increased. Lately, incubators have produced much more data than ever. This may stem from the improvement of entrepreneurial activities, with help from technological improvements, and from an increasing number of graduated companies that continuously produce data. Additionally, the newest types of incubation, such as accelerators, also help produce data. Hence, future studies will have more tangible data for researchers to examine and to develop more common facts.

In this study, we provide a final set of benchmarks for the 33 chosen KPIs. These benchmarks are aggregated from a literature review after an extensive search. However, the final set of benchmarks is susceptible to quantitative research. Such research with a multiple criteria decision analysis would be a significant contribution to BI assessment literature. Additionally, this paper created a general list of benchmarks ignoring the industry differences in BIs since there are not yet enough benchmarking studies on BIs. A benchmarking framework for each industry such as biotech, chemistry, agriculture and information technology would be a great addition to studies on BI assessment.

References

Find references here:

Co-authors: Laura Peconick, Vinicius Sobreiro, Herbert Kimura, Josep Pique