A Defense of Peer Review - by Asimov Press and Dan Elton

I used to see peer review as primarily good — an important gatekeeping process, essential for protecting capital “S” Science from all manner of quackery and nonsense. Looking back, I was clearly naive; plenty of junk science gets published in top journals despite the supposed guardrails of peer review. Given the replication crisis, an exponentially growing deluge of low-quality or fraudulent work, and growing evidence that science is stagnating, some have been wondering if we should get rid of peer review altogether and try something different.

One prominent essay forcibly arguing against peer review is “ The Rise and Fall of Peer Review,” by Adam Mastroianni. In it, Mastroianni calls peer review a “failed experiment” and concludes that the best path forward is to do away with it altogether. He suggests that peer review doesn’t “do the thing it’s supposed to do” — namely, “catch bad research and prevent it from being published,” so perhaps scientists would be better off uploading PDFs to the internet where review and oversight can happen post-publication. 1 Mastroianni isn’t exactly sure what a post-peer review world would look like, but in follow-up essays he encourages experimentation. Similar sentiments have been expressed elsewhere, such as by Brandon Stell, the founder of PubPeer, a website where anyone can leave comments about academic articles.

My experience at the receiving end of peer review has been positive; I’ve co-authored 42 peer-reviewed articles, mostly in the fields of physics and artificial intelligence for healthcare. But despite my good track record with peer review, it is clear that the existing system has serious flaws. We have all heard stories of high-quality work getting rejected while mediocre or error-laden work slips through (numerous high-profile fraud cases come to mind). It’s also undeniable that we lack enough data to say whether we get sufficient returns from peer review relative to the enormous amounts of time, resources, and energy we invest in it. Overall, issues such as these are why I found — and continue to find — many of Mastroianni’s arguments compelling.

Leave a Comment