Writing a Review

Much has been and is still being written about the deteriorating quality of research publications, and perhaps research itself. One important factor here is the job done by the reviewers. Of course, in many so-called conferences, there is no serious process of reviewing, and almost papers make it to the ‘accept’ list so that there will be enough people attending the conference. And this leads to another evil that no one comes to a conference to listen and learn, but only to talk/present, often to a scarce audience who is only waiting for their turn! But that is the matter for another post.

A while back, I was track chair for an IEEE event, and we had a bunch of papers for my track. We located some academics in and around Mumbai, who had reported expertise in the area, and we sent 2-3 papers to them. Most of the reviews that came back shocked me! Here are some samples.

a) Paper is written well. It can be more detailed considering the scope of the title. Minor formatting issues are found.

b) 1. Some grammatical mistakes are found. 2. References need to be written properly. 3. Paper is well written in terms of content.

c) orientation: contents must have proper spacing. Use of passive language should be maximized. Diagrams must have proper indent/fonts.

contents: More literature survey is required. Detailed algorithms(names)are to be mentioned.Results must be shown before conclusion. References must be in proper format.

This is what comes as the comments to authors. What does an author do with these kind of comments? It does not even give any feel if the papers is to be accepted or rejected, and why.

Reviewing a paper is an academic commitment, and key to maintaining standards of research work. It also helps new researchers grow recognising their weaknesses and correcting them. I have often seen reviewers who just tick ‘good’ for most of the evaluation parameters, and leave no comments, and tick ‘accept’. I think these are criminal acts from you as a researcher/academic.

The fault may not be yours completely. Often under pressure for institutions to produce PhDs and faculty to become PhDs, we ignore almost all of what constitutes research and what constitutes academic value system. This creates a vicious cycle of degrading research quality. You either return a favour by being very liberal in evaluating another’s paper, or follow what you have seen others do to your paper. You contribute to the downward spiral knowingly in the former case, and unknowingly in the later case. Time to shed a tear or two...

There are plenty of material on how to review a paper. There are etiquettes of what to do and what not, and how to do and how not to. These need to be part of our research methodology curricula – where else do they learn these from, in a system that lacks proper eco-system? And for the same reason, I am not repeating all that. But let me touch on the key ingredients of a review.

To review a paper, you must make a serious attempt to understand what the author is trying to say. If this is easy to do, the paper is well written. If it is too difficult, the paper is too badly written. And usually most papers fall somewhere in between – you need to spend some energy in decoding the intentions. It is a good idea to summarise what you understood in this process as the beginning of the review. It shows the author that you have made an attempt to read the paper, and also provides a base to see if you have understood his intentions correctly. And it also provides a cushion before you enter the ‘review’. This can be as long as a sentence or two, to a paragraph. No need to reproduce the paper here!

Now you have a base for your review. Time to ask questions. Does the work make sense overall? Is it a good problem? Is the problem relevant to the journal/conference you are reviewing for? Of course, for conferences like ‘international conference on computing, communication, electronics and mechanics’, any topic under the Sun or beyond are ok. You should, perhaps, stay away from such “conferences”!

Now on to the core of the paper. What is the problem the authors are addressing? Have they described their approach clearly? Is the approach appropriate in your view? Has enough details of the approach been provided? What is their test target? Is the test properly designed? What are the results? Do they make sense? Are the results reliable (look at sample size, variance, influence of other parameters, etc)? Is there anything odd in the results? Have the authors noticed and attempted to analyse any anomalies?

Does the paper review relevant literature? Do you see some literature that is missing? Is the literature reviewed relevant to the chosen problem and chosen approach (if known)? What inference is the paper making from the literature? Is the author’s approach adequately different from the approaches in the literature? In what way?

As you can see a reviewer would have many questions of this nature. It is your job to ask as much of these questions as possible, and where you feel something worth sharing, add to your review. This can be a concern (“you have not addressed ...”), a query (“how will your system react if ...”), a piece of information (“have you seen the work of ...”, “have you tried the .. approach”), and so on. Don’t forget to appreciate if you find some part well done.

Formatting issues, proper use of English, consistent naming conventions, etc are also important. But that comes only after the content review. Point out specific examples of errors. Comments like “improve English” does not help anyone.

Reviewing involves work, and should not reduce it to a glance at the paper and ticking a few options. But it is a good work – it is a good exercise for your brain. It is what gives health to the academic community.

When phrasing your comments, be mild and constructive. They, usually, are learners and look for guidance from you. Of course, there are a few who intentionally flout norms usually because they want to hide their ignorance, or the fact that they have nothing to say. I feel, giving some strong dose for them, is ok. But you may differ. It is useful to think that one day someone will do to you, what you are doing to someone else today.

Taking the reviewing job lightly allows a lot of junk to come into the system, and degrades the overall quality of the event. It also gives a false sense of happiness to the authors, who loses a chance to improve themselves. Writing a paper is not a one-day job. It takes much effort, thinking, revision, etc to make a good paper. Casual efforts must be discouraged. Authors who are not willing to put in efforts should also be discouraged. Reviewing is not optional for an academic. It is part of our commitment and a key ingredient in sustaining ourselves. You must do, and you must do it well.

-0-