A Rubric to Evaluate Preprint Peer Reviews

Disclosures: This rubric was created collaboratively by undergraduates at Mount Holyoke College (MA, USA) as an assignment in a course taught by Dr. Rebeccah S. Lijek, Assistant Professor of Biological Sciences. All authors declare no conflict of interest. 

Context: This rubric was created by undergraduates enrolled in a course on peer review, in which students learned about the peer review process and then wrote peer reviews on preprints. Peer review is considered by many to be crucial to the scientific process. Preprinting allows researchers to publish manuscripts on free, online preprint servers before or instead of through a journal’s peer-review process. Preprinting offers many advantages, yet there is little guidance available as to how preprints could be peer reviewed. Our class aimed to demystify the peer review process for undergraduates, centering discussions of equity and thinking critically about bias, with the goal of bringing more people into the peer review process. Since preprint peer review does not have barriers to access in the same way that journal-curated peer review does, it allows for all members of the scientific community to participate including early career researchers and educators. We created this rubric to help reviewers think about how a review’s structure, style, and substance can create an effective and humane critique. We hope that use of this rubric might improve the quality of the review and so the corresponding manuscripts. The rubric could be used by reviewers themselves to self-evaluate their review before publishing it and/or by educators who incorporate peer review assignments into their classrooms. We welcome your feedback, adoption, and/or adaptation of this rubric.