You can also search for this author in PubMed Google Scholar Since th

ChatGPT is transforming peer review — how can we use it responsibly?

submited by

Style Pass

2024-11-10 05:30:04

You can also search for this author in PubMed Google Scholar

Since the artificial intelligence (AI) chatbot ChatGPT was released in late 2022, computer scientists have noticed a troubling trend: chatbots are increasingly used to peer review research papers that end up in the proceedings of major conferences.

There are several telltale signs. Reviews penned by AI tools stand out because of their formal tone and verbosity — traits commonly associated with the writing style of large language models (LLMs). For example, words such as commendable and meticulous are now ten times more common in peer reviews than they were before 2022. AI-generated reviews also tend to be superficial and generalized, often don’t mention specific sections of the submitted paper and lack references.

That’s what my colleagues and I at Stanford University in California found when we examined some 50,000 peer reviews for computer-science articles published in conference proceedings in 2023 and 2024. We estimate that 7–17% of the sentences in the reviews were written by LLMs on the basis of the writing style and the frequency at which certain words occur (W. Liang et al. Proc. 41st Int. Conf. Mach. Learn. 235, 29575–29620; 2024).