Testbeds. Survey results.

Post 2 in a series of posts from the project on testbeds, public procurement, and regulations commissioned by the Ministry of Economic Affairs and Employment.

Last year, we at Upgraded were commissioned by the Ministry of Economic Affairs and Employment to study the experiences of Finnish health startups in three areas: testbeds, public procurement, and regulations. We jumped at the opportunity to investigate these topics separately and in connection with each other. At this point, when asked what areas they were finding difficult to tackle, our members would name one or all of them.

The report turned out to be extensive; in this post series, we would like to break it down by topic. This time, we will share the survey results on the topic of testing and validation. 14 out of 35 respondents had experience with testbeds.

The majority of the respondents in our survey (64%) have used the services of a testbed two or three times, with only seven percent testing in this setting once, and only one company coming back to a testbed more than five times. It shows that testing in the healthcare and wellbeing industry is an iterative process; although not all companies choose to go that route, those that decide to use the services of a testbed become recurring customers. It is interesting to note that the testbed location does not directly correspond to the company’s headquarters.

Not all companies chose to share the cost of their testing projects. Those that did, shared an amount between zero and 600,000 euro with the median of 7,500 euro. The company reporting an amount of 600,000 euro said that most of it is due to patent cost. It must be noted that two companies that took part in the survey indicated that the testbed services they used were free, and we even had one that was paid a thousand euros to perform testing by their ordering customer.

Overall, companies in our sample had neutral or good experiences with testing and validation (see the agreement graph). Among the major challenges that they encountered while working with testbeds, we found:

• complex information flow and communication in a multi-stakeholder project,
• lack of consolidated information on the various testbeds, and information on differences between those,
• a slow delivery process,
• the staff’s workload preventing them from dedicating themselves fully to the
• project,
• the vicious circle of not being able to start validating the solution without certification and not being able to get certification without a testing and validation process complete,
• the lack of scaling strategy after the pilot,
• the lack of financial support after the pilot,
• the delays in project realization due to the Covid-19 pandemic, and
• culture barriers.

Although two of the fourteen respondents were very critical and said that they could not name any advantages to the existing testbeds in Finland, most have been able to list several. Startups characterize Finnish testbeds as cooperative and relatively easy to communicate with; they note that project implementation is straightforward and not bureaucratically burdened, and that testing facilities deliver what was promised.

Testbed staff’s attitude and expertise have been praised, as well as the quality of the obtained data and high credibility of the findings. Low cost, availability of funding, and information support from Business Finland have been mentioned as one of the key positives, although this opinion is not universal. References, networking, and contacts have been also listed as an added value of using a testbed in Finland, as have been the direct reach to real patients or customers and the collaboration side of the testing projects. Respondents convey that the feedback and development ideas they received were appreciated, along with more practical details such as location and working times.

You can read the report in full here.

Shoot an email to our Head of Research, Alena, if you have any questions about the study: [email protected]

The series Table of content:

Post 1. Introduction.

Post 2. Testbeds. Survey results.

Post 3. Testbeds. Interview analysis. Expert perspective.

Post 4. Testbeds. Interview analysis. Startup perspective.

Post 5. Testbeds. Roundtable summary.

Post 6. Public procurement. Current state of affairs.

Post 7. Public procurement. Survey results.

Post 8. Public procurement. Interview analysis. Expert perspective.

Post 9. Public procurement. Interview analysis. Startup perspective.

Post 10. Public procurement. Roundtable summary.

Post 11. Standards and regulations. Current state of Affairs.

Post 12. Standards and regulations. Survey results.

Post 13. Standards and regulations. Interview analysis. Expert perspective.

Post 14. Standards and regulations. Interview analysis. Startup perspective.

Post 15. Standards and regulations. Roundtable summary.