- This retrospective study of 5,013 manuscripts submitted to JAMA Network Open investigated whether peer review metrics changed with the onset of the COVID-19 pandemic
- The pre-pandemic period studied was January 1, 2019, to March 10, 2020; the pandemic period studied was March 11, 2020, to June 29, 2021
- The mean rate of reviewer acceptance of review requests remained stable (39.5% pre-pandemic vs. 38.4% during the pandemic; P=0.21), and the rate was substantially greater for COVID-19–focused manuscripts (67%)
- Mean turnaround time was slightly but statistically significantly shorter during the pandemic than before (14.4 vs. 15.8 days; P<0.001), and for COVID-focused manuscripts it was 13.7 days
- The quality of review improved somewhat; the mean number of reviews per manuscript that editors rated as very good or excellent increased from 1.3 before the pandemic to 1.5 during the pandemic (P<0.001)
The COVID-19 pandemic rekindled conversations about the viability of peer review, especially its burden on researchers, how best to ensure continued rigor and value of reviews and how to cope with the high volume of COVID-19–related manuscripts submitted by investigators who expected rapid publication.
Subscribe to the latest updates from Psychiatry Advances in Motion
Roy H. Perlis, MD, MSc, director of the Center for Quantitative Health in the Department of Psychiatry at Massachusetts General Hospital, and colleagues found the quality of peer review did not diminish during the pandemic at JAMA Network Open. In fact, their report cites a slightly shorter turnaround time during the first year of the pandemic and modestly increased editor-reported review quality.
The researchers retrospectively examined JAMA Network Open peer review data from two time periods:
- Pre-pandemic—January 1, 2019, to March 10, 2020 (14.3 months)
- Pandemic—March 11, 2020, to June 29, 2021 (15.6 months)
They included all 5,013 manuscripts received that were categorized as Original Investigations (n=4,295) or Research Letters (n=718) for which the editors sought at least one content review. 1,860 and 3,153 manuscripts were submitted during the pre-pandemic and pandemic periods, respectively.
On most measures, there were significant changes in the means before and after March 11, 2020:
- Number of manuscripts reviewed per week—30 pre-pandemic period vs. 46 during the pandemic (P<0.001)
- Number of reviewers invited per manuscript to achieve the minimum number of required reviews—6.0 vs. 7.0 (P<0.001)
- Proportion of reviewers per manuscript who accepted invitations—39.5% vs. 38.4% (P=0.21)
- Number of reviews returned per manuscript—1.6 vs. 1.7 (P<0.001)
- Number of reviews per manuscript rated by editors as very good or excellent—1.3 vs. 1.5 (P<0.001)
- Time to return reviews—15.8 vs. 14.4 days (P<0.001)
In multivariable linear regression adjusted for article type, study design, and direct submission versus transfer, time to return reviews remained shorter in the pandemic period (mean difference, −1.2 days; 95% CI, −0.7 to −1.6).
Results for COVID-19–focused Manuscripts
Similar patterns were observed when comparing manuscripts that concerned COVID-19 (n=915) with those that did not (n=2,238):
- Number of reviewers invited per manuscript—6.4 vs. 7.2 (P<0.001)
- Proportion of reviewers per manuscript who accepted invitations—67.2% vs. 64.8% (P=0.006)
- Number of reviews per manuscript rated by editors as very good or excellent—1.5 vs. 1.4 (P<0.001)
- Mean time to return reviews—13.7 vs. 14.6 (P=0.002)
The peer review process at this large open-access journal continued to function well during the pandemic despite changes in the volume of submissions and, presumably, many reviewers' work and home environments. Still, continued study of medical peer review is needed because there's abundant evidence COVID-19 has impacted researchers negatively, especially women.