Behaviour change 101: How to do a Rapid Review

Authored by Peter Slattery (with support from Peter Bragge)

In this ‘Behaviour change 101’ series, we draw on our experience and educational resources to discuss approaches for planning and executing behaviour change research and projects.

In our work at BehaviourWorks Australia (BWA) we are frequently asked ‘What does the research say about getting audience Y to do behaviour X?’. When our partners need an urgent answer we often provide it using a Rapid Review. In this article I explain Rapid Reviews, why you should do them, and a process that you can follow to conduct one.

What is a Rapid Review?

Rapid Reviews are “a form of knowledge synthesis in which components of the systematic review process are simplified or omitted to produce information in a timely manner” [1]. Indeed, with sufficient resources (e.g., multiple staff working simultaneously) you can do a Rapid Review in less than a day. The outputs of these reviews are, of course, brief and descriptive, but they can be very useful where rapid evidence is needed, for example, in addressing COVID-19.

Rapid Reviews can therefore provide detailed research within reduced timeframes and also meet most academic requirements by being standardised and reproducible. They are often, but not always, publishable in peer-reviewed academic journals. 

Specifically, while a Rapid Review follows similar protocols to a Systematic Review (e.g., establishing a search strategy and setting inclusion and exclusion criteria) it is differentiated from conventional systematic reviews by virtue of one or more deliberate simplifications, such as:

• focus on a narrow topic that will return few search results
• use of short time frames when searching (e.g., only searching for studies published in the last 2 - 5 years) 
• use of relatively few databases
• exclusion of certain types of literature (e.g., primary studies) 
• use of only one researcher (rather than two) to screen and extract results
• extraction of relatively few fields of information from the paper
• simple rather than complex synthesis of results
• limited or no quality assessment of included reviews 
• creating short or simple outputs.

At BWA we frequently start with ‘a review of reviews’ approach. If no reviews are available, recent high-quality primary studies are sought. The subsequent report provides a quick way to access and understand the most high-quality findings within a particular area. Our Rapid Reviews are often accompanied by Practice Reviews - which provide “'real world' insights from those involved in developing and/or delivering strategies to address the problem under investigation.

Why do a Rapid Review?

The short answer is that review-level evidence is the most robust form of research to inform decision-making. Good decision making requires an understanding of context and causality. This is perhaps especially challenging when making decisions about behaviour change because behaviour occurs across a multitude of different contexts and has complex, varied and interacting causes which differ over time and across actors. 

Research is one of many inputs into 'good decision making' and different types of research provide different qualities of input. The best research input provides a summary of the full body of relevant research. This minimizes risks that can come from making decisions based on considering a single study in isolation; for instance, the risk of choosing an intervention that other research shows to only be effective within a particular lab setting or audience. 

For example Figure 1 [source] illustrates how the field of Evidence Based Medicine ranks the quality of different types of research evidence. 

It shows that Evidence Based Medicine regards Meta-Analytic and Systematic Reviews as the gold standard for research evidence; two systematic review techniques which involve analysing secondary data (e.g., checking and comparing multiple studies). 

However, the rigour of such research comes at a cost to its rapidness. Meta-Analytic and Systematic Reviews can require extensive resources and take up to two years to complete. As Peter Bragge, Associate Professor & Director of Health Programs at BehaviourWorks Australia and Lead of the Monash-McMaster Social Systems Evidence Collaboration points out, “governments are often unable to commission a Systematic Review due to time constraints”. 

As a result, he notes that relatively ‘Few policymakers, decision-makers and media are using Systematic Reviews to respond to complex challenges. Instead, they are searching Google, and hoping that something useful will turn up amongst an estimated 6.19 billion web pages."

If there is insufficient resource, or impetus, to do a Systematic Review, then a Rapid Review is the next best option and one that is considerably better than searching Google! These reviews can provide a systematic overview of a body of research and are easier to complete within short timeframes. 

What are the limitations of Rapid Reviews?

Of course there are limitations to Rapid Reviews that should always be made explicit. While Systematic Reviews and Rapid Reviews generally result in similar conclusions [3], the expedited nature of a Rapid Review creates an increased risk of not identifying relevant literature or insights that would be gained from a full review. Thus one can be more confident in the results of a longer review process.

Additionally, for complex issues that are not easily or immediately solvable such as obesity and chronic disease management, it can be more efficient in the long term to do one ‘living Systematic Review’ (i.e., a systematic review that is kept up to date) rather than repeated Rapid Reviews over long time periods, or a tool like Social Systems Evidence. 

What is needed to do a Rapid Review?

In most cases, doing a Rapid Review will require research training and expertise. If you do not have this training or the time to learn then we recommend consulting an expert, particularly if producing a review with significant implications. 

If you proceed with doing a review then make sure that you have a university, library or personal subscription so that you will have access to i) relevant research databases, ii) the publications you retrieve from them and iii) software and processes needed to complete the review process.

How we conduct Rapid Reviews

The following section briefly summarises six stages that BWA uses to do Rapid Reviews.

Preparation

• Plan the type of Rapid Review to be undertaken (e.g., see the SelecTing Approaches for Rapid Reviews (STARR) Decision Tool).
• Identify and refine a specific topic, research question and set of aims (consider using the PICO framework).

Develop and refine the: 

• search terms (e.g., terms to search for) 
• strategy (where to search, who to contact, what types of literature to include)
• inclusion and exclusion criteria
• databases that will be searched (e.g., Scopus, PubMed)
• quality appraisal tools to be used (e.g., AMSTAR 2 for systematic reviews)
• extraction plan and template to collect data from the relevant papers.
• Pilot test initial ideas, for example, by testing different searches on different databases to get the ideal sensitivity and specificity, then revise plan and timeline as needed.
• Finalise a protocol (i.e., write a plan for the search that describes the elements above)
• Register the protocol using a service such as PROSPERO or OSF.

Search

Conduct searches on the selected databases and, 

• evaluate search yields and revise due dates if necessary
• download the records returned into research software such as Endnote, Zotero or Mendeley)
• de-duplicate records if needed
• import database files into screening software (e.g., Covidence).

Screening and selection

Conduct title and abstract screening:

• screen paper title and abstracts based on your inclusion and exclusion criteria 
• resolve disagreements between those doing the screening (if relevant).
• Conduct full-text screening:
• download full-text files for all relevant articles
• screen in screening software
• resolve disagreements.

Quality appraisal

• Conduct quality appraisal on the relevant papers, for example, using something like an excel spreadsheet.

Data extraction

• Extract data from all relevant papers using extraction template and relevant software.

 Synthesis and write up

• Generate a PRISMA diagram to summarise your review.
• Generate readable extraction tables.
• Write a narrative summary of findings using the extraction tables.

Recent Rapid Reviews that BWA have completed

In 2019 we completed a rapid overview of reviews of Research co-design in heath. On OSF, you can see the protocol that we registered and our materials, in addition to the extraction and quality assessment tables. Extraction and quality assessment were conducted in Excel then later converted into word documents.

Other recent Rapid Reviews that BWA has undertaken include:

1. Digital Inclusion & Health Communication: A Rapid Review of Literature. 
2. Interventions to promote healthy eating choices when dining out: A systematic review of reviews
3. What interventions could reduce diagnostic error in emergency departments? A review of evidence, practice and consumer perspectives

Summary

Rapid Reviews enable governments and other organisations to gain a rigorous understanding of current knowledge in weeks, rather than waiting months or longer for full Systematic Review through simplifying aspects of the Systematic Review process (e.g., reviewing reviews rather then primary studies) while following the key phases of this processes (e.g., preparation, searching, screening and extraction).

In our training we teach about Rapid Reviews and other methods of evidence synthesis. We can also help develop evidence synthesis and storage capacity within organisations.

Please get in touch if you think that we can help you. Email: [email protected]