By: Juliana Ancalmo, Chad E Cook PT, PhD, FAPTA, Ciara Roche

Background

Critical appraisal is a hallmark of peer reviewed publishing. Critical appraisal provides analytical evaluations of whether the results of the study can be believed, and can be transferred appropriately into other environments, for use in policy, education, or clinical practice [1]. Historically, critical appraisal is performed by peer reviewers who are either content or research experts (or both). Peer reviewers have viewed this act as an obligation for science, especially those who benefit from peer review as authors, and are not currently paid for this service.

Recent limitations brought forth by qualified peer reviewers has ignited discussion around paying for reviewing services. Although this topic had been highly debated previously, a new wave of conversation was reignited when researcher and Chief Scientific officer James Heathers [2] argued for a $450 fee for a peer review in an editorial published on Medium. This, coupled with the challenges many researchers faced post-COVID have spurred people on both sides of this argument to speak out. In this blog we will outline the pros and cons of this debate and discuss the complexity of the issue at hand.

Pros of Paying Peer Reviewers

We propose several benefits of paying peer reviewers for their critical appraisals. Since the COVID-19 pandemic, there has been a notable decline in acceptance rates combined with an increase in submission rates in academic journals, creating a large imbalance within the peer review process [3]. Compensation could lead to reviewer buy-in and decrease this imbalance [2]. Interestingly, it may also increase the diversity of peer reviewers. Peer reviewers often reflect those who populate their field of study, which often are dominated by men. Theoretically, paying for peer review may better represent women and lower income countries, especially if they are targeted [4].

On top of a lack of diversity of reviewers, these publishing companies are generating record profits, and compensating reviewers may reduce the associated negative optics. For example, the arguably biggest academic publishing company in the world is Elsevier, which generates $3.35 billion in revenue with a profit margin of around 40% [5]. It is arguable that among the five major publishing companies, Elsevier, John Wiley & Sons, Taylor & Francis, Springer Nature and SAGE, who control 50% of all revenue of the academic publishing industry globally, a solution could be drawn up to financially compensate underpaid and worked reviewers [5]. Quite frankly, asking someone to do a lot of work for free is a tough sell during times of record profits. Finally, we believe reviewers simply deserve to be paid. Good reviewers spend a lot of time peer reviewing papers. This process improves the final manuscript and strengthens the science. Experts deserve to be compensated and asking people to work for free is an archaic and offensive stance.

Cons of Paying Peer Reviewers

There are also several arguments that can be made against paying peer reviewers. One often cited is that compensation of reviews may lead to unethical reviews being submitted. It is not a stretch to consider how reviewers may take advantage of this monetary system for their own financial benefit–this could impact the quality of the reviews submitted, as reviewers work to do as many reviews as they can for “easy cash-grab.” This leads into another concern regarding the payment of peer reviewers: there is currently no threshold on what constitutes a “good review.” Nowadays it can take several months to wait for feedback on a paper, only to receive a couple of lines from a reviewer and a rejection from the editor. Does this two-line review deserve the same compensation as someone who spent hours reading and giving critical feedback on a review?

It is clear there would need to be notable training and standardization in submitting a review that would qualify for compensation; however, this process would further limit individuals who could submit a review and may cause further delays in this process. Additionally, processing of the payments would likely be a disaster at first. Considering it can sometimes take a year for these journals to publish a review, it is not unreasonable to believe that a payment system for ongoing peer reviewers would result in lost, incorrect or delayed payments.

Finally, there is uncertainty of whether journals or the industry could even afford to pay these reviewers in the first place. Publishing consultant, Tim Vines, argued that if there is an average of 2.2 reviews per article, each reviewed article would cost $990, assuming the $450 fee proposed by Heathers is met [6]. Additionally, for a journal with a 25% acceptance rate, the cost of reviewing for each accepted paper would be $3,960 [6]. This additional cost would almost double research journals expenditures, which may lead journals to increase article-processing charges and subscription fees to cover these additional expenses.

Our Thoughts

In theory, we support payment for peer review. However, the traditional practice of peer review may be resistant to change due to system inertia, which is the resistance of an organization to change despite its necessity [7]. We support the need for additional steps before a model flip can occur. These include reducing unnecessary burden on reviewers such as: 1) papers that have fatal flaws and have no chance of acceptance; 2) requests that are outside the scope of the reviewers; 3) multiple requests at one time; and 4) unrealistic review turnaround. Simply put: there are too many submissions–a focus on quantity over quality. Predatory journals and journals that support weak science are interested only in publishing (and article processing fees) and little on science. We acknowledge the complexity of institutional reform in the presence of system inertia. Once these elements are sorted, we can get back to the discussion of paying for peer review.

References

  1. Katrik P, Bialocerkowski AE, Massy-Westropp N, Kumar VS, Grimmer KA. A systematic review of the content of critical appraisal tools. BMC Medical Research Methodology. 2004;22:(4).
  2. Heathers, J. The 450 Movement. Medium. Available at: https://jamesheathers.medium.com/the-450-movement-1f86132a29bd
  3. Künzli N, Berger A, Czabanowska K, Lucas R, Madarasova Geckova A, Mantwill S, von dem Knesebeck O. I Do Not Have Time -Is This the End of Peer Review in Public Health Sciences? Public health reviews. 2022;43. https://doi.org/10.3389/phrs.2022.1605407
  4. Cheah PY, Piasecki J. Should peer reviewers be paid to review academic papers? Lancet, 2022;399(10335):1601.
  5. Curcic D. Academic Publishers Statistics. WordsRated. Available at: https://wordsrated.com/academic-publishers-statistics/
  6. Brainard J. The $450 question: Should journals pay peer reviewers?. Science https://www.science.org/content/article/450-question-should-journals-pay-peer-reviewers
  7. Coiera E. Why system inertia makes health reform so difficult. BMJ (Clinical research ed.), 2011;342:d3693.