How do you manage bias during peer review?

Despite its fundamental importance to the scientific system, there are only a few empirical studies regarding the peer-review system and these have shown little consistency between judgments and demonstrated evidence of regular rejection of studies that have won major prizes, including the Nobel Prize. The high error rate in predictions of future significance calls into question the suitability of "prophetic" peer review assignments in modern science. It should be reconsidered whether the question of significance should be completely removed from the peer review process and only the technical and methodological correctness of the work should be assessed.

It is almost impossible to not sympathize more one school of thought with respect to another. Therefore, in my opinion, a researcher cannoy have a completely objective point of view by nature of his scientific orietation. Once this limit accepted, a reviewer is conscious that to turn her judgement into an objective analysis has to give more space to the research results belonging exactly to those ideas and hypothesis which do not belong to her convictions. This way could be minimized the gap and possibly reach a good level of objectivity.

In academic peer review, maintaining objectivity is paramount to ensure the integrity and quality of the evaluation process. And so, for me, my role as a peer reviewer is pivotal in upholding the standards of academic integrity and progress. Recognising and managing bias is an essential part of my responsibility. Fostering self-awareness, embracing transparency, seeking diverse perspectives, and maintaining constructive feedback practices, will contribute to a more robust and fairer peer review. Ultimately, no matter my bias, my efforts contribute to advancing knowledge and the credibility of scholarly discourse.

We are emotional by nature, because we are humans. So, we need a mechanism to control ourselves being bias at peer reviews. Before every peer review season, I read the vision and mission of the organization. I have created a form to be filled by reviewer. I have included the line "Do not base on your friendship or conflict. Your input here is a valuable indicator to improve the operation. Thank you". This reminds the reviewer about the assigned role. Review is 75% qualitative and 25% quantitative. Objective is to improve the process only.

Create a small checklist to keep your bias radar consistent. It can include things like: 1. Is a group overrepresented or underrepresented? 2. Do the methods themselves introduce bias (such as selection bias)? 3. Is the article written for too narrow of an audience? 4. Does the article conform to traditional trends rather than exploring the full scope of the topic? 5. Are there overly-influential stakeholders (such as funding sources)?

I created a peer review rubric for our journal to ensure consistency in peer review. It still allows for a high degree of reviewer autonomy and input, but it also forces reviewers to address key areas like "sources of bias."

Example of Editor bias: Early in my career, I submitted a paper for publication in a physics journal. It was quickly rejected by the editor. I work at a community college and suspected bias. I reached out to the expert who I discussed my findings with and he advised that I revise and explicitly acknowledge our discussion in a the resubmission. The editor, who was US based, rejected the paper again in less than 24 hours. I then approached the situation differently: I changed the paper's title and started a new submission, choosing an editor not based in the US( I had that option) My hope was they wouldn’t hold any biases against community colleges. The editor finally sent the paper to the reviewers. After revisions, the paper was accepted!

"Bias" isn't a dirty word. We must acknowledge that we all have biases and can't fully eliminate them. However, when we are aware of sources of bias, we can do our best to negate their effects on our work.