Australian innovation metrics and a 2024 update to systems and platforms for mapping and measuring innovation ecosystems
Chad Renando
Supporting Australian entrepreneurship and innovation, Research Fellow UniSQ, CEO Startup Status, MD - GEN Australia
Austrade recently announced that it is adopting the Dealroom platform to showcase venture investments and startup activity in Australia. The announcement provides an opportunity to explore Australian approaches to innovation metrics and evaluation and reflect on progress over the past several years.
Towards a national approach to innovation metrics in Australia
In 2020, on the back of supporting the release of the Australian Global Entrepreneurship Monitor (GEM) report in Australia , I wrote a series of posts including:
My post on the metrics review 2020 concluded that "I feel there is a tipping point coming over the next one to two years in terms of advances in innovation metrics."
I was overly optimistic.
A national, strategic, aligned approach to innovation metrics and measurement appears to be elusive.
James Riley 's media company InnovationAus.com documented the four-year effort to develop a national innovation metrics approach, including:
The other reference to this work is in the Industry Innovation and Science Australia (IISA)'s 2022/2023 annual report, which includes a brief statement outlining the metric project's progress.
In terms of internal government data, the three-year, $130.8 million Data Integration Partnership for Australia (DIPA) project ended in 2020. The Australian Government Digital Transformation Agency's 2023 Data and Digital Strategy focuses on internal systems but does not reference specific areas such as innovation metrics.
Progress has been made at the state level. LaunchVic manages evaluation in Victoria in collaboration with partners, including Deloitte Economics, Dealroom, and Tech Council Australia. Queensland government's Advance Queensland publishes detailed metrics about the impact of its programs, including state of innovation reports and subsets such as regional and commercialisation topics. New South Wales through Investment NSW has created the NSW Innovation and Productivity Scorecards since 2018 as a a snapshot of NSW’s innovation and productivity performance in comparison to other states and selected international economies. A platform for Western Australia is developed through the Spacecubed Foundation with support from Lottery West.
When looking at private sector solutions, Dealroom is working its way through state-based installations alongside the recent national release, but they appear unconnected at this stage. Startup Muster appears to be making a resurgance in its second year following a four-year haiutus. Current private innovation data sources in Australia include Techboard , Ramenlife, Startup Muster , Startup Status , Decode (Innovation Ecosystem Analytics & Network) , Cut Through Venture , and Startup Galaxy.
Solution principles
There is a need for a shared, common approach to innovation metrics and evaluation in Australia.
We acknowledge that the challenge appears resistant to change for a few reasons:
Keeping a solutions-focused perspective, here are some considerations for a path forward:
1. Clear leadership
We need clear and consistent leadership in the area of innovation metrics. This includes ongoing support that is not dependent on the three-year swing of the political pendulum. This support needs to be cultural - embedding metrics and evaluation as a way of being that is as important as program delivery; financial - ensuring evaluation is accommodated in program funding; and structural - establishing a dedicated institution or organisation to be accountable for national innovation metrics across domains and jurisdictions.
2. Clear ownership
The review of current progress indicates that there may be a lack of clarity on the appropriate custodian of innovation metrics in Australia. There needs to be a dedicated agency, institution, or organisation that can integrate and engage with appropriate stakeholders and data sets, be trusted and have appropriate governance to manage innovation data, have the capability and capacity to respond to dynamic needs, and provide national direction and global representation for innovation metrics. This would be expected to be funded by government with a vested interest, but with autonomy and the ability to integrate with academia and industry.
3. Public and private solutions
The solution needs to leverage government data such as ABS and BLADES, primary and secondary academic research, industry data including engagement with peak bodies and Research Development Corporations, and private sector solutions such as Startup Genome, Dealroom, and other platforms outlined in the review below.
4. Inclusive and integrated datasets
The solution needs to accommodate a broad view of innovation and acknowledge social and community indicators alongside economic outcomes. Innovation has a tendency, and some might say an inevitability, to centralise wealth in developed economies. A metrics platform that rewards rapid growth through rankings without reporting on correlating increases in disadvantage can do more harm than good.
5. Establish variables where consensus is difficult
A challenge experienced in previous approaches is arriving at a shared definition of common metrics, such as a "startup" or "innovation". Rather than getting hung up on arriving at a consensus, find the most common element to be agreed upon, such as a "firm", and then create variables for different contexts, such as the firm's age based on industry or technology enablement based on regionality.
What's next
These reflections are based on my work in innovation ecosystems over the past decade. My current personal contribution is through work with the University of Southern Queensland in developing evaluation approaches and supporting regional place-based development, the not-for-profit Startup Status mapping and measuring the Australian innovation ecosystem, Global Entrepreneurship Network Australia , and the place-based initiative Ready Communities .
I remain involved in national conversations around aligning metrics. My current focus is on immediate impact to support individual programs, build capacity in industry bodies, develop evaluation models for place-based initiatives, and publish results to advance the conversation in both academic and practice forums. It will be interesting to see what has changed when I post again in four years.
As always, feedback, updates, insights, and collaborations are welcome as we collectively work to advance opportunities for everyone in Australia.
Now, on to the review...
A 2024 map update
The considerations and limitations from the previous post still apply concerning accuracy and completeness, category cross-over, audience, and recommendations and critique.
For consistency, we will use the 2020 categorisation for the 2024 review:
You can refer to the previous post for detailed commentary on the categories. This 2024 review will focus more on changes in Australian approaches over the past four years. References may be cursory if a platform has not significantly changed.
I am not looking extensively for new developments in global solutions that may be emerging but are not on our radar as yet. The review is also based on observations and personal experience, some of which may be open to interpretation and bias.
Corrections, updates, and links to new approaches are always welcome.
1.0 Surveys
Global
The two main global surveys continue to be the Global Entrepreneurship Monitor (GEM) and Startup Genome. These surveys are different in methodology and outcome and there is value in running both approaches in parallel.
The Global Entrepreneurship Monitor was last delivered in 2019 . As a randomised phone sample survey, GEM is as close as we can get to a representative sample of a population's views on entrepreneurship. The approach includes an adult population survey (APS) which is a randomised phone survey to the adult population and a national expert survey (NES) which is a smaller survey to select experts in the country. In 2023, the GEM report is based on a contribution of 46 countries that completed the full APS and NES and three more countries that only completed the NES.
Startup Genome focuses on a smaller population sample at the city level. Results are shared in the annual Global Startup Ecosystem Report (GSER) which ranks countries based on metrics including valuation of company exists and investments. The public website allows a comparison of around 170 ecosystems. I released an in-depth analysis of the methodology and application for Australia from the 2024 GSER , including rankings of Australian cities Sydney, Melbourne, Brisbane, Adelaide, Canberra, and Perth.
Two surveys noted in the 2020 review have not had recent updates. The Global Accelerator Learning Initiative (GALI) has not had a report released since 2021, and the InBIA IMPACT Index survey was last released in 2016. The member-based UBI Global World Rankings of Business Incubators and Accelerators continues with the last report published in 2023, and Startup Heatmap Europe continues to collect data with the open survey that has been running since 2016.
Australia
In Australia, the prominent survey focused on the startup sector is Startup Muster . At the time of the last review in 2020, Startup Muster had not been delivered since 2018. The survey was brought back in 2023 and is now active in 2024.
The Venture Capital and Later Stage Private Equity (VC&LSPE) Survey referenced in 2020 has ceased, with the last run in the 2018-2019 financial year.
The Australian Bureau of Statistics (ABS) Business Characteristics Survey (BCS) continues with an open survey . Results are listed on the ABS Innovation in Australian Business website . The most recent 2022-23 results are based on a random sample of approximately 7,000 businesses via online forms or mail-out questionnaires with a response rate of 82.5%.
2.0 Mapping and Representation
In the 2020 report, we categorised mapping and representation platforms as national commercial platforms, national bespoke platforms, global examples of local maps, and Australian national and local approaches. The lines blur as national platforms provide bespoke solutions and smaller custom approaches attempt to scale.
The Dealroom platform had 11 instances in the 2020 review. This number has grown to over 75 as of 2024, including an Australian national installation and state-specific platforms in New South Wales, Queensland, Victoria, Western Australia, and South Australia. It is not clear if there is any coordination between national and state platforms. The full list of Dealroom installations is provided in an appendix at the end of this report.
Startup Nation represents and analyses ecosystem data for Israel. The platform was previously provided to other economies, including New Zealand and Thailand. None of the installations listed in the 2020 review are still active, and there is no reference to other implementation options on the Startup Nation website.
Canada-based Hockeystick was founded in 2014 and sold in 2023, with founder Raymond Luc now acting as a senior advisor at Deloitte. Hockeystick had a brief foray in Australia, including support for a local accelerator.
领英推荐
Decode Systems, founded by Aresian Investments, also appears to be no longer operational. The website is no longer active, the product site is semi-functional , and the last LinkedIn post was from 2021.
Other maps referenced in the 2020 report that are no longer active include Startup Europe Map and Welsh Arloesiadur. In Australia, the national InnovationMap from CSIRO's Innovation Catalyst Collaborative Research Centre (CRC) project is not active. Victoria's Melbourne Laneways redirects to LaunchVic likely due to being superseded by the adoption of Dealroom. Other local examples Innovate Noose and Geelong's GeeMap are not available.
US-based SourceLink provides a suite of services and products, including SourceLink Pro CRM, Resource Navigator, and a SiteConnex portal. Ecomap , also based in the US, continues to expand their services.
The London-focused TechLondon continues to reference as being powered by Gust but appears to no longer be associated with IBM . It is a list of news and events related to the London Tech Scene.
The Founder Institute's Startup Ecosystem Canvas continues to provide bespoke maps, with 132 maps listed.
In Australia, the RamenLife platform continues to support TechSydney . The site shows recent events and job postings and a representation of support programs, with the most recent news from 2020.
Full disclosure: I am the director of the not-for-profit company Startup Status and the underlying mapping platform.
I continue to develop the Startup Status platform, which now tracks around 5,000 support roles in the Australian innovation ecosystem. The platform is expanding into policy analysis and has been used to support data for startup support research in NSW , research organisations including AgriFutures, industry bodies including AusAgriTech , and government initiatives including the Queensland Decarbonisation Hub . I also provided a representation on the Austrade website, which also may be superseded by the national Dealroom implementation.
3.0 Government data
Continuing the approach in the 2020 review, we focus on innovation platforms owned and supported by government data. We note other countries such as the European Innovation Scorecard (EIS) and, while not government, the Consumer Technology Association's US Innovation Scorecard .
In Australia, the Australian Innovation Monitor has a legacy from 2000 but was last updated in 2021. Excluding Dealroom implementations, other state department approaches noted previously include:
Government data is available through the Australian Bureau of Statistics (ABS) with direct access through the Business Longitudinal Analysis Data Environment (BLADE) environment. BLADE is accessed through institutions and individual projects . There are 294 projects listed from 2017 to 2023, including 156 projects with Commonwealth Government departments, 114 with Academic Institutions, 21 with State and Territory governments, 3 Private Institutions and 2 Local Government.
4.0 Integration projects
Startup Commons continues its work on an integration platform for systems that focus on startup business creation, managing support services for startups or managing & coordinating ecosystem development, for sharing and distributing applications between ecosystems around the world. The website lists products, including Orchestra For Ecosystem Services Mapping, Skyview Ecosystem Insights, EcosystemOS, and Ecosystem Advisor AI.
Integration projects in Australia referenced in 2020 included Longitudinal Australian Business Integrated Intelligence (LABii) out of QUT and the SA BLADE system managed out of the Entrepreneurship Commercialisation and Innovation Centre.
5.0 Index and Ranking
The Intelligent Community Forum (ICF) continues to provide the annual Intelligent Community Awards. Between 2020 and 2024, Australian cities listed in both the Smart21 Communities of the Year and the Top7 Intelligent Communities of the Year include Adelaide, South Australia; Prospect, South Australia; Sunshine Coast, Queensland; Townsville, Queensland; and Greater Geelong, Victoria.
The Organisation for Economic Co-operation and Development (OECD) maintained data sets including:
The last World Economic Forum Global Competitive Index was released in 2020 . Related, the Insead Global Talent Competitiveness Index ranks countries against six areas of Enable, Attract, Grow, Retain, Vocational and Technical Skills, and Global Knowledge Skills. Australia ranks 8 out of 134 countries.
The World Intellectual Property Organization (WIPO) produces the Global Innovation Index (GII) . Australia ranked 24 out of 132 economies in 2023, similar to the ranking of 22 in 2019.
Other rankings referenced in 2020 include the Bloomberg Innovation Index and the Kauffman Foundation Indicators of Entrepreneurship . Not a ranking, but the MIT Regional Entrepreneurship Acceleration Program (REAP) is worth noting for ecosystem development.
Startup Blink continues to provide rankings. There was question as to the validity in the 2020 review, noting the one sample of Brisbane, Queensland ranked at position 87 and increasing 31 positions from 2017 to 2019. In 2024, Brisbane is ranked 102nd in the Startup Blink index. A cursory glance at the Startup map shows the company Growing up in NZ as the number one startup in Glen Innes, NSW, and appears to be a direct pull from Crunchbase, including any errors in the data. Caution should be used when considering Startup Blink results.
Startup Genome is also listed in index and ranking based on the annual city-based rankings in the Global Startup Ecosystem Report.
The Startup Meter index referenced in 2020 is no longer active. The Regional Australia Institute INsight online interactive map of Regional Competitiveness in Australia is also no longer available.
The only Australia-specific indexing platform noted at the time of the review is the Investment NSW Innovation and Productivity Scorecard providing a snapshot of NSW’s innovation and productivity performance in comparison to other states and selected international economies. A digital scorecard will be released in 2024.
6.0 Business insights / Profiles
There has not been much change in the portfolio of global insights platforms, which include Crunchbase , CB Insights , Owler , Pitchbook , Mattermark , Traxn, and dun & bradstreet.
The DueDil platform now redirects to the FullCircl CRM platform. The platform Vest noted in 2020 may not be maintained.
As noted in my 2024 GSER review , there can be significant discrepancies and quality issues within and between platforms. This is shown by the March 2024 review using Australian startup Goterra as an example:
This should be kept in mind when relying on indexing systems such as Startup Blink or Dealroom that may directly reference data from other platforms without a substantial curating and cleaning process in place.
In Australia, Techboard continues to focus on "Aussie Startup and Tech data", with an emphasis on regular investment reports. A new addition from 2020 is the Cut Through Ventures report which provides reporting on investment data.
We are also seeing sector-specific profiles supported by industry groups. Examples include the directory of climate-tech providers with Climate Salad , the agritech ecosystem with AusAgritech , agrifood innovation with the AgriFutures growAg platform , and the Sunshine Coast manufacturing ecosystem with Manufacturing Excellence Forum Sunshine Coast.
7.0 Additional functionality
MetaBeta continues as a management platform for accelerators and incubators to track dealflow, provide metrics and reports, and manage mentor engagement. A new addition The Biz Lab provides related functionality for empowering incubators with data-driven insights.
Gust continues its focus on investment, providing deal management tools for angels and helping startups through the business creation process. Microtracker remains as a set of data tools and services that allow microenterprise development organisations?to self-assess and improve.
f6s continues to support the application process for hackathons and accelerators, lists jobs, and provides other benefits to founders such as credits and discounts. The platform still has challenges with outdated data.
Angelist changed name to Wellfound and focuses on jobs. Startup Space referenced in 2020 is no longer active. Funderbeam has a specific focus to allow you to trade investments or buy stakes in a selection of ambitious companies across international markets, listed on the Funderbeam Singapore Marketplace. CoVentured sold in 2024 and is now Singapore-based and is an AI-enabled innovation discovery platform.
8.0 Region analytics and research
The global companies still stand as:
As do the Australian platforms:
9.0 Other mapping tools
Other mapping tools include:
2024 update reflections
The ecosystem of innovation evaluation approaches and platforms is maturing. We will see a continued rationalisation of platforms and consolidation in one or two global systems.
There is a growing number of institutions interacting with the central government BLADE data platform. These research projects need to have greater public access under a coordinated leadership that also integrates the number of diverse data sources across the nation.
There are also a growing number of datasets managed by industry groups to support data-driven advocacy. This should be encouraged as strong industry bodies provide better representation of the diversity necessary for innovation.
The other change not reflected in the data is the growing number of leaders and data experts across industry, government, and academia focused on innovation metrics. I encourage national direction towards innovation metrics but I meter my optimism based on the past four years. That said, the continued increased in talent, passion, and awareness within individuals has me looking forward to new solutions and innovative approaches for addressing the challenges with innovation evaluation in Australia.
Appendix: Dealroom instances
Director of Innovation; Associate Professor at UTS: Transdisciplinary School
2 个月Great post as always Chad! We completed a major project on innovation metrics at the level of Innovation Districts for Greater Cities Commission that's making its way through a few Investment NSW groups. Hopefully mcuh more to come!
President Victorian Bioenergy Network, Co-founder & Chair Gaia EnviroTech & Gekko Systems, Independent Director Agnico Eagle, Founder and Patron CEEC International
2 个月Gaia EnviroTech
Growing B2B Businesses from Start- to Scale-ups to Corporations through Operations, Growth Strategies and Marketing.
2 个月Brilliant, Chad!
Innovative Real Estate Developer at Savills | Transforming Visions into Reality
2 个月Intriguing update! How would harmonized metrics benefit diverse ecosystems' growth trajectories? Curious about your thoughts. Please send me a request so we can chat. ?? Chad Renando
Climate technology advisor, investor and supporter
2 个月Huge post Chad. Nice work.