Fewer papers, more research

Researchers and editors all know the problem: The number of submitted papers is increasing. It is hard to get reviewers for journal papers. Even researchers that accept review requests often do not perform them. Authors are then left waiting an agonizingly long time and chasing editors, who in turn chase reviewers.

Paper evaluation is and will remain a very labour-intensive industry’ Currently, the game after submission is unfair. Reviewers do a lot of work and receive few tangible benefits. Authors receive huge tangible benefits for accepted papers, but there are very few risks associated with rejected ones.

Such a system incentivizes hyperproduction (including salami slicing and even unethical behaviour, such as submitting the same paper to two journals), many resubmissions (after rejection, an author can do minor changes and resubmit the paper to another journal), and the ‘lottery approach’ (even if I think the paper is nothing special, I have an incentive to first submit to an A+ journal, then two A journals and only then to a B-ranked journal). So, the same paper is reviewed five times for five different journals. Not only that; a lot of reviewers’ efforts are lost in this process (the authors might decide to ignore excellent suggestions if they would require too much extra work).

Simply put, the number of submitted papers is too large. So, for journals, the core solution is to design incentives to decrease the number of submitted papers.

Several possibilities exist:

1.      Money: I am a full professor at a business school, so this is the obvious idea: charge submission fees to the authors, then pay the reviewers (e.g., 400$ submission fee for a journal, 100$ for each reviewer, 100$ for the editor). The editor would need to approve every payment to make sure the reviewer put in sufficient efforts to warrant payment. Reviewers could use the earned money as ‘vouchers’ for their submissions or have it paid out in cash.

1.1.   With the voucher system, everyone could submit for free after (s)he (or a co-author, e.g., thesis supervisor) completed four high-quality reviews. Of course, somebody may not be competent to serve as, e.g., an MISQ reviewer. However, in such a case, it is implausible that (s)he is capable of writing a paper with any chance of MISQ acceptance.

1.2.   Such a system would lead to a redistribution of money, especially in favor of the editors. However, one could argue that this is fair.

1.3.   My central concern: this would turn reviewing from a social/communal transaction (‘I review to help others/the editor/the community’) into a monetary transaction. It is possible that this would decrease the availability of reviewers. 100€ for reviewing a paper that goes through, e.g., three review rounds is extremely low for the amount of work involved.

2.      Different ways of paper acceptance: Change it from the final binary decision (accept or reject) to accept papers at, e.g. B, C and D ranks. Then, a top journal could accept a good (but not top-notch paper) with a C rank. The rank would be clearly stated on the paper along with the open explanation about strengths and weaknesses of the paper (e.g., ‘great idea and model but rigorous testing is missing‘).

3.       Make rejections public: Currently, everyone can see my published papers. Nobody (except my wife) knows the number of my rejections. Making rejections public would make me consider whether to submit the paper or work to improve it. This would be easy to do: Each journal would require a unique ID (e.g., ORCID, Scopus ID, Researcher ID) and then an aggregator would compile the list of my rejected papers (maybe even with a summary of the main reason for rejection of each paper).

3.1.   I understand that this might be a highly controversial idea and could lead to public shaming (even more so if the long list of rejections would be visible to the general public). Researchers all know that rejections are part of the business. I am unsure how the general public (or my students) would react if they saw some of my nastiest rejections. But still, all defeats of a tennis player are public as well (Roger Federer lost 261 games in his career).

4.      Require transfer of reviewers’ comments: If a paper is rejected from one journal, one could submit to another, but the prior reviews would be attached (along with author’s response to the previous review). In the current system, it is entirely up to an author whether (s)he will consider the comments of reviewers/editors of the rejecting journal. (S)he can just submit the same paper to a similar journal, hoping for better luck.

5.      Incompetent (or late) reviewers are a huge problem. Reviewers’ rankings (done by the editors or even the authors themselves) could be another (controversial) idea. Then, it would be public how many high-quality reviews somebody had written. Publons is a step in this direction.

6.      Require more transparency/less salami slicing: All papers from the same case/dataset should be explicitly listed as a part of the same project (ResearchGate allows that). Authors often ‘play’ with constructs, making up to 10 papers from the same dataset and mass-mailing them to journals. Consider a recent Ph.D. graduate who turns his Ph.D. into eight papers, each rejected an average of four times (two were published in solid journals); the workload for editors/reviewers to handle 32 papers is huge.

6.1.   If the journal would require authors to list previous rejections/parallel submissions, I think most people would follow the rule. It is impossible to ensure that everyone would do it. But, it is always possible for somebody to fabricate an entire dataset. However, all researchers I know would consider data fabrication to be highly unethical. Many of them would, however, be fine with writing five similar papers from the same dataset hoping that at least a few would be accepted.

Most readers of this post are now shouting, ‘Peter, you got it all wrong. You are missing the main problem: the whole business model of the publishing business is wrong’. After all, the fact that researchers write ‘papers’ and ‘submit’ them to ‘journals’ where they are ‘peer-reviewed’ and finally ‘published’ is a very peculiar (and by no means the only possible) way to disseminate new ideas. However, that is already another question.