Testing AI-enhanced reviews for Linux patches
Code review is in high demand, and short supply, for most open-source projects. Reviewer time is precious, so any tool that can lighten the load is worth exploring. That is why Jesse Brandeburg and Kamel Ayari decided to test whether tools like ChatGPT could review patches to provide quick feedback to contributors about common problems. In a talk at the Netdev 0x18 conference this July, Brandeburg provided an overview of an experiment using machine learning to review emails containing patches sent to the netdev mailing list. Large-language models (LLMs) will not be replacing human reviewers anytime soon, but they may be a useful addition to help humans focus on deeper reviews instead of simple rule violations.
I was unable to attend the Netdev conference in person, but had the opportunity to watch the video of the talk and refer to the slides. It should be noted that the idea of using machine-learning tools to help with kernel development is not entirely new. LWN covered a talk by Sasha Levin and Julia Lawall in 2018 about using machine learning to distinguish patches that fix bugs from other patches, so that the bug-fix patches could make it into stable kernels. We also covered the follow-up talk in 2019.
But, using LLMs to assist reviews seems to be a new approach. During the introduction to the talk, Brandeburg noted that Ayari was out of the country on sabbatical and unable to co-present. The work that Brandeburg discussed during the presentation was not yet publicly available, though he said that there were plans to upload a paper soon with more detail. He also mentioned later in the talk that the point was to discuss what's possible rather than the specific technical implementation.
