THE DOCTORATE SERIES (3 OF 7): THE (SYSTEMATIC) LITERATURE REVIEW

“The most essential factor is persistence – the determination never to allow your energy or enthusiasm to be dampened by the discouragement that must inevitably come.” – James Whitcomb Riley

In this third blog in my informal guide to pursuing a doctorate (DBA), I will focus on what I felt was the first really big deliverable; the Literature Review or as we referred to it during our program the Systematic Literature Review (see David Denyer’s work for more detail).

I’ll tackle this blog in the following sections: (1) What is a Systematic Literature Review (SLR), (2) What question does it address and how is this different from your research question, (3) How do you do an SLR (what are the steps), (4) How do you use it or what is the point, and (5) A few practical tips that helped me. 

WHAT IS A LITERATURE REVIEW

If you have read my first two blogs you will see how I position the literature review as the step you take before finalising your research question (what you want to find out/answer by undertaking empirical research). Crucially, and this was confusing to me for a while, the purpose of the literature review is to address a related but separate question to your research question (RQ). What you are simply trying to answer is what literature already exists that may: (1) inform your research; (2) clarify whether someone has already attempted to address a similar question; (3) see how (which methodologies) they pursued their research; and, (4) make sure you know who are the main players/authors in your field.

In essence, you step back from your pursuit of a narrow research question (RQ) and do a database search on key words in literature. The idea is to capture the intersection of major literature themes (i.e. leadership and change or family business and change). A good way to illustrate this exercise is by using a Venn diagram as follows:

Figure 1 – Venn Illustration of Targeting Literature Intersections

In my Figure 1 illustration, I am just painting a picture of how my own enquiry could be viewed as the overlapping space between 2, 3, or 4 literature domains. It should not be a surprise that the target is narrow. You are after all looking for something undiscovered or something specific which you want to look at differently.

There are no rules on how many intersecting literature domains you can search for but obviously you want to try to identify any literature which someone at some stage might say “Ah….but have you considered what Joe Blogs said on this subject!?” These gotcha moments are your ultimate nightmare should you be unprepared for them at Viva and your job here is to try to avoid them.

SYSTAMATIC LITERATURE REVIEW

Just a quick word on ‘systematic literature review.’ This is a specific methodology for undertaking a literature review. It is advocated by some universities (including Cranfield). The idea is you show painstaking rigor and discipline in searching existing literature. Since this is what I did, let me explain the steps:

FIRST YOU DEFINE YOUR SEARCH WORDS

In my case, I did the following:

My draft research question was: ‘What are the enablers for and barriers to planned change in family owned businesses?’ I then debated how to extract from this the major literature domains which might inform the question. I debated this with as many people who were prepared to listen. As I did, I drafted an SLR search enquiry as follows: How and when does planned/strategic change occur in family owned businesses?’ I then broke that question out as follows:

This question might be deconstructed into 3 key components:

1.    Change – and appropriate synonyms

2.    Family – as above

3.    Business – as above

CONSTRUCTING A DATABASE SEARCH

My next step was to convert the above words into search strings for database searches. Fortunately, I had the help of some wonderful people at my school library. They helped me get to the following first draft of search words:

Figure 2 – Illustration of database search strings

A key point here is that this exercise was iterative. I experimented with these search strings several times before getting comfortable they gave sensible results. One of my tests was to hand search key journals and use Google Scholar to identify what looked like important papers. I then tested whether these had been captured by my database search.  Fundamentally, you want confidence that you have cast a good net and that all of the obviously important literature is in there. You of course hope to get some pleasant surprises as well.                                                                   

SELECTING DATABASES TO MINE

Once you’ve got some search strings you start to experiment with electronic libraries like API, EBSCO, and Scopus. There is no perfect way to do this as many libraries have similar materials. However there are some differences and you are expected to try to optimise across at least three sources. Here were my initial results:

Figure 3 – Initial Database Results

I heard stories of 10k plus database searches and others with less than 1,000. The important point is that you want to feel confident you have: (1) captured the key literature but that; (2) you have not given yourself an impossibly large set of literature to interrogate. You quickly find out that this process results in a great deal of rubbish getting collected. You don’t want to waste too much time on needlessly ploughing through off topic papers. Achieving a reasonable (defensible) balance is your goal.

THE 6 STEPS OF AN SLR

Once you’ve got your database search, you follow these next 6 steps in a very disciplined way, tracking as you go the decisions you make and reasons why you made them. This all becomes critical when you are ultimately faced with explaining to an examiner how you can assure them you considered all the pertinent literature:

Step 1 – Capture your literature search results (I transferred these to Mendeley and then into an Excel file to be able to sort them and make selection notes against each title).

Step 2– Remove duplicates and obvious errors (despite my attempts to filter them out I got some posters and non-academic papers. You will also get duplicated papers from each database).

Step 3– Conduct a title review of all the papers that make it this far. This will be a lot. I did this and was able to take out about 20% of the papers. You have to be careful because your bias must be to leave papers in but I had titles on inheritance tax planning and even China’s one child policy which were easy to reject).

Step 4– Abstract review: This stage requires some investigative work as sometimes your database search won’t have pulled down an abstract. Bottom-line though, this is just a slog of reading 1,000 plus abstracts and deciding what to include/reject. The critical point I would say is to have written criteria on what to include. Here the goal is to ensure, as far as possible, you are including all plausible papers. But, of course, you realise in doing so that you are committing yourself to read the whole paper as a next step...so you want to be sure it will be worth your time.

Step 5– Full paper review: You hope by this stage you have the core literature addressing your area of interest. You are now sitting down to read what you hope is the collected/cumulative wisdom on your reserach area. However, the sad reality is about half of what you read will probably still be off topic. Consequently, while the question may be more delicate here than at Step 4, you will still make decisions to discard papers either because they are not on topic or do not meet a minimum quality benchmark. I must say I found the second criteria of quality debatable and was inclined to include all peer reviewed articles. 

There is no other way to explain this step than to say…take a deep breath….get them all downloaded in Mendeley on your iPad…and just read. Every time you complete a paper make a decision to include or reject. Use a ‘quality’ checklist and diligently keep these to show anyone who wants to see them. Examiners will definitely want to know how you made these decisions. I also made notes on the paper, key words, and conclusions in Mendeley and then transferred the notes to my excel master database when I next had my laptop handy.

Step 6– “Bowtie” and Citation Mapping: This last step is intended to make sure you haven’t missed anything. For me it included 3 pieces: (1) I looked through all the citations of the papers I had included and completed steps 3 through 5 above for each of them; (2) I asked my supervisors and other contacts whether I was missing anything and got several referrals which I also put through their own steps 3 to 5; and (3) I went to the main journals my subject was published in and did a hand search (this means just searching all papers published in specific journals for key words and checking they have been covered by your SLR) of them to see if I missed anything.

This whole process is tedious, boring in parts, but completely necessary. It took me the best part of 6 months. I was very pleased when it was finished.

WHAT’S THE POINT OF THE SLR

The exercise I’ve described probably sounds painful and it should! The point however is to make sure you know before you undertake any empirical research what has already been investigated. Once you’ve completed the SLR, you will have:

1 – Confidence you’ve read 99% of the probable research into your field

2 – Better understood what existing scholars currently know/believe about your field of enquiry

3 – And…this is the key outcome of your literature review….you then get to reposition your research question (RQ) based on your analysis of what is already out there.

I found that once I’d read, analysed, and critiqued the available literature I was able to probably for the first time state my specific area of enquiry and how this might potentially be a discovery…in other words…something that no one else had previously looked at.

Just one final word on the analysis and suggestion of finding a gap. Don’t worry about this. It will come. I was nervous that I wouldn’t find anything unique or new. In reality, after 6 months reading the literature, I found a dozen things I could have focused on. I chose the one which appealed to me most and was closest to my original motivation. It worked out fine!

A FEW CONCLUDING TIPS

As I reflect on doing my SLR at Cranfield, and particularly with my Viva fresh in my mind, let me share some quick tips that made a difference for me. I have tried to make these short and (semi) sweet:

(1)  Research Question (RQ) versus Literature Search (SLR)– This confused me and perhaps it might you. I understood this much better in retrospect. The key lesson for me is that the literature review helps you define and lock down your research question (RQ). It is however a broader search of literature and starts with agreeing with your supervisors what the key literature domains might be and the key words you should use.

(2)  Use your Library Resources– Cranfield’s librarian was amazing! I had 3 skype calls and several emails with her (thanks again Mary!). She essentially sat with me while I did the first 3 or 4 attempts at mining different databases. Without her help I would have been lost. If you have a Mary at your school, go see her and be very, very nice to her.

(3)  Software– You have to have a system to track what you are doing, keep notes, and sort through the literature. I used the following:

a.    Mendeley to hold all the PDF’s and References. I also used this to read and keep notes. You can highlight text and, critically, there are tabs or labels you can add to papers which I used to say “accept’ or “reject.”

b.    Excel as a filtering and tracking database. I transferred all the references into excel and used standard filtering and graphing tools to report into my paper. I found you could transfer files from Mendeley to Excel if you used JabRef as an interim stage.

c.     Have selection criteria and stick to it. Write down what your inclusion criteria is. Print them out as cover sheets (Step 5) . Fill one in for every one of your final papers. 

d.    Back up your data. I had one nightmare moment when I’d reviewed about 30 papers and thought I’d lost my notes. Use multiple back-ups, save your data, and make sure your devices are charged.

(4)  Read all your papers before forming a narrative. I found it was easier to complete the reading tasks and then form in my mind how I would categorise them. I found in the end that there were 4 ‘buckets’ of literature which each had a different perspective on my subject. I summarised these views and then explained where I thought there were gaps.

(5)  Remember this is an exercise in refining your RQ and proving you have a new perspective. This is like a detective novel. You read the papers and try to find undiscovered angles. Obviously this is the point but it is surprisingly easy to forget that and just focus on summarising what has already been said (as Admiral Ackbar might say... it's a trap!).

(6)  Head down…persevere…..you eat a whale one bite at a time. The SLR is an exercise in stamina and discipline. There are no short cuts. It will take XX hours. You get done when you’ve put those hours in. For me, this was only enjoyable when you’d finished it.

To read more of my blogs click here to visit my website.