The culture change of the REF 2028
UK Research Integrity Office (UKRIO)
UKRIO gives confidential advice and guidance on good research practice in academic, scientific and medical research.
Many aspects of the proposed new look to the UK’s Research Excellence Framework (REF) are welcome and to be applauded. As a veteran of the REF and its predecessors going back to the early noughties, I can say that the research assessment process has evolved almost beyond recognition.
Particularly welcome is evolution of the focus away from the selection of individuals with a certain number of ‘outputs’ to the unit – I remember the stress and distress of the days when staff were informed formally whether they were submitted or not, and the fear of the impact this would have on careers and livelihoods. ?When this was initially relaxed in REF 2014, an elaborate and somewhat intrusive process was introduced whereby staff could be exempted from submitting four outputs if they could provide evidence of personal circumstances that prevented them from producing four outputs within the REF census period. Whilst further clarification on aspects of the new proposals is needed, the emphasis on the unit rather than the individual is a step in the right direction in my view.
I also welcome the increased emphasis on the research environment and culture, and I look forward to seeing the outcome of the commissioned work to develop appropriate indicators, and of the consultation that has been launched into this aspect of the new system. UKRIO will be responding to this consultation and encourage others to do so as well. It was a surprise and disappointment that such an important change was not included in the initial consultation.
I have two niggling worries. The first is that it will be very difficult to develop appropriate and comprehensive indicators of research culture in the rich and varied research environments that we are fortunate to have in the UK (to declare an interest, I work in a small, specialist institution as well as for UKRIO). Work has been carried out on this in the past, for example ‘Indicators of Research Integrity – An Initial Exploration of the landscape, opportunities and challenges’, which highlights some of the challenges ahead in the development of a comprehensive set of indicators that will work well for all institutions and disciplines.
The second is that in the process of attempting to assess research culture, something will change. Institutions are doing great work in this area and the enhancing research culture funding that has been available for the last couple of years in parts of the UK has supported this work. Quite apart from the nuts and bolts of how the assessment will be achieved, I believe there is a danger that the assessment process itself could lead to a change in mindset, from being focussed solely on developing and gradually improving your research culture to one where there is an awareness that the work being done will be judged and measured and a rating put on it, and that will impact the REF outcome for the institution and the funding resulting from it.
领英推荐
That could change what people do and how they do it. It could bring in an element of competition that by-and-large has not been present before, resulting in less sharing of good practice and resources. This is not to say that work on research integrity and research culture should not be subject to review and impact measurement, but there is a difference between self-evaluation and improvement, and external assessment that impacts on future reputation and funding. Such incentives can have mixed effects, as shown by a landscape study that UKRIO carried out with partner organisations.
An immutable feature of the REF that will not change is its ability to have unintended consequences and to impact institutional behaviour in ways that are not foreseen when the rules are drawn up. I hope that a version of the ‘Observer Effect’ does not occur following the introduction of a formal assessment of research culture.
A healthy research culture safeguards and enhances quality and ethical standards, and thereby helps ensure trust in research. I and my colleagues at UKRIO look forward to responding to the consultation on this key aspect of the REF and using our unique experience and perspective to inform the process of its development.
Written by Nicola Sainsbury , Research Integrity Manager at UKRIO and Research Manager ROSE BRUFORD COLLEGE OF THEATRE AND PERFORMANCE .
Reader in Condensed Matter Theory Group Leader, Physics of Quantum Materials University of Kent
1 年"An immutable feature of the REF that will not change is its ability to have unintended consequences and to impact institutional behaviour in ways that are not foreseen when the rules are drawn up" A sharp observation!
Philosopher | Research Ethics & Integrity | Global Data and AI Ethics, Policy, and Governance
1 年Thank you Nicola Sainsbury for this considered analysis. I agree that fundamentally much comes down to the research environment and culture. We also need to consider the diversity of research institutions, including their manner of education and the impacts they strive for, on education, on science, on the humanities, on society. Surely, we are now at a critical juncture as the tools and demands of our digital societies play a greater and greater role and the frameworks for research integrity are being developed.
Academic, consultant, ethicist and policy advisor. Championing diversity and inclusion in knowledege-based decision making
1 年My main concern with whatever version of the REF is implemented is that it will always and invariably reduce things that are multidimnesional and complex (whether it is impact, culture, or research excellence) to a simple checklist of unidimensional indicators that will never capture the full extent of what people do. Checklists also incentivise box-ticking attitudes and behaviours, often to game the system. I saw this at the last REF, where the requirement to submit only research active academics prompted many universities to move those perceived as low performing individuals to teaching only positions! What builds research cultures is often unmeasurable - all the unseen and unrewarded work of building relationships, offering advice to a junior colleague, checking over a protocol (or ethics application), giving your place in the spotlight to someone less advantaged, giving a leg up to those that are below you. These little things over time create collegiality, improve cultures and, ultimately, make better research. But how do you measure them? The focus away from individuals is good, but let's see how units are measured. If it's done by adding up individual outputs, then we are not really moving away from individuals, are we?