Help! I'm in a R&D system of systems- get me out of here!

Musing about system effects on collaboration at an organisational level, I started to realise that the UK R&D 'system' is, in fact a system of systems.

A single researcher or group sits in an academic department, that sits within a university or research centre, which sits inside of a local innovation ecosystem, a national research and innovation funding system and a global research and innovation system. Universities also sit within their places, their communities and broader society.

That same researcher or group (or university or department or funder or government) will be subject to influences from, and in turn influence, each of these systems and the way they interact with each other.  This could make you feel very small and insignificant and that you can't change anything. In fact, the opposite is true and I'm going to try and explain why.

The UK R&D system of systems

The UK R&D roadmap[1] published in 2020 defines the “R&D system” as:

‘what connects universities, research institutes, government labs, charities and businesses to each other and to sources of funding. It supports the UK’s wider ecosystem of public, private and third sector organisations to push the boundaries of knowledge and turn great ideas into economic, environmental and social benefits.’

The R&D system in the UK is really a system of systems (SoS). This is a huge subject which emerged from the field of systems engineering and is now being applied much more broadly to socio-technical and socio-economic systems in various settings. A helpful overview of systems of systems can be found at the ‘Guide to Systems Engineering Body of Knowledge, SEBoK[2]. In the taxonomy of types of systems of systems (SoS), we can think of the UK ‘R&D system’ as a collaborative system of systems where “the component systems interact more or less voluntarily to fulfil agreed upon central purposes. The central players collectively decide how to provide or deny service, thereby providing some means of enforcing and maintaining standards”. The central players in our system include, but are not limited to, the UK Government, research funders, universities, large companies and publishers.

Just as we want to optimise an organisational system around agreed goals, the UK R&D roadmap sets out a desire from the UK Government to optimise some the features of our R&D system, highlighting in particular the need for this SoS to be an ‘efficient and effective system that enables strategic decision-making, provides the right incentives to enable research and innovation to inform each other, and ensures that money flows to the best researchers, innovators and entrepreneurs with the least friction possible,’ and that ‘we should look to ensure that [the system] is coherent and efficient, with the right incentives for institutions to collaborate and not duplicate’.

Within this system, which as we have already seen is actually a system of systems, there are high levels of connectivity and the potential for emergent behaviour. This means that each participating organisation is able to look for leverage points which might shift the dynamics in ways which it considers desirable. That's the good news. The bad news is that emergent behaviour means that an intervention that you expect to have a significant impact might not, and vice versa. Some of the interventions that result are more effective than others. With hindsight we can look back and see that some interventions in the UK R&D system have been effective, and can also have lead to unexpected and undesirable outcomes. A great example of this is the Research Excellence Framework (REF) in the UK. The UK REF is used to allocate nearly ￡2 billion in annual research funding. It uses peer assessments and research outputs across the UK as metrics for measuring success.

In around 2009, the then UK Research Councils (RCUK) introduced a new assessment criterion for the review of research proposals, to encourage researchers to be actively involved in thinking about how they would achieve ‘excellence with impact’. Impact was broadly defined and included both academic impacts and socio-economic impacts. While researchers complied with the new requirements in terms of their proposal, this change in policy did not result in a significant cultural change within universities. What did lead to universities taking the so-called ‘impact agenda’ seriously, was the inclusion of impact as part of the UK Research Excellence Framework (REF) in 2014.

Around 20% of a UK university’s institutions income generated by REF 2014 would now be down to the quality of its impact case studies, and in total around ￡1.6bn of public funds would be awarded on the basis of impact in the UK over the next 5 years[3]. This leverage point was so much more powerful than the one operating at a single grant level, and not just because of the sums of money involved, although that certainly was one factor. This leverage point worked at multiple levels in the cultural web. It engaged with the people with real power to make change within an organisation. The stories of impact, also known as impact case studies, became a way for the universities to communicate the value of their research to a broader audience outside of academia, and to different parts of their own organisation. Universities introduced control systems to identify and select the case studies that would be submitted as part of REF, and having a case study selected would be a source of prestige and therefore a symbol of your status and success. 

The inclusion of impact in the 2014 REF is a great example of how REF can lead to positive culture change within the R&D system, Unfortuanely we also know that some other aspects of REF have not had such positive outcomes. Amanda Solloway, the UK Science MInister, highlighted the importance of REF on the UK R&D system of systems in her recent speech[4]. She said, “There are now very few parts of academic life in the UK that are not affected in some way by the REF. The REF ruleset, implemented in a risk-averse way, has become the default tool for many university leaders to effect institutional change. But a risk-averse compliance culture risks stifling creativity and diversity.We risk breeding resentment and eroding trust in our ability to evaluate the system effectively and fairly.”

She went on to say that “we must be prepared to look to the future and ask ourselves how the REF can be evolved for the better, so that universities and funders work together to help build the research culture we all aspire to.”

The challenge for the minister, and for all participants in our UK R&D system of systems is that it is complex and not all of its properties will be predictable and in line with the expected outcomes. The cultural web may be a useful tool for looking at any proposed changes as it provides a multidimensional perspective of the cultural paradigm.

But what does this mean for us as individuals? The good news is that as participants, what we do as part of this system of systems matters. Small changes can have big impacts, particularly when carried out by lots of participants, consistently, over time. As the saying goes, ‘Be the change you want to see in the world’[5].

If we want to create a more collaborative, diverse and creative system, what we do matters.  And we can start today in whatever way we can. We may each be small but by picking our leverage points and acting with integrity and passion we can make a difference. That is the power of systems of systems.

[1] https://www.gov.uk/government/publications/uk-research-and-development-roadmap/uk-research-and-development-roadmap#being-at-the-forefront-of-global-collaboration

[2] Mike Henshaw, Judith Dahmann, Bud Lawson " Systems of Systems (SoS)" in SEBoK Editorial Board. 2020. The Guide to the Systems Engineering Body of Knowledge (SEBoK), v. 2.2 R.J. Cloutier (Editor in Chief). Hoboken, NJ: The Trustees of the Stevens Institute of Technology. Accessed [27-10-2020]. www.sebokwiki.org. BKCASE is managed and maintained by the Stevens Institute of Technology Systems Engineering Research Center, the International Council on Systems Engineering, and the Institute of Electrical and Electronics Engineers Computer Society. https://www.sebokwiki.org/wiki/Systems_of_Systems_(SoS)

[3] https://www.kcl.ac.uk/policy-institute/assets/ref-impact.pdf

[4] https://www.gov.uk/government/speeches/science-minister-on-the-research-landscape

[5] This is often attributed to Mahatma Ghandi. In fact, Ghandi said “We but mirror the world. All the tendencies present in the outer world are to be found in the world of our body. If we could change ourselves, the tendencies in the world would also change. As a man changes his own nature, so does the attitude of the world change towards him. This is the divine mystery supreme. A wonderful thing it is and the source of our happiness. We need not wait to see what others do.” ‘Be the change you want to see in the world’ is a nice precis, or paraphrasing, of the quote.