Ruff v0.4.0: a hand-written recursive descent parser for Python
Ruff is an extremely fast Python linter and formatter, written in Rust. Ruff can be used to replace Black, Flake8 (plus dozens of plugins), isort, pydocstyle, pyupgrade, and more, all while executing tens or hundreds of times faster than any individual tool.
This release marks an important milestone in Ruff’s development as we switch from a generated to a hand-written recursive descent parser.
Parsers form the foundational layer of any static analysis tool, transforming raw source code into Abstract Syntax Trees (ASTs), which serve as the basis for analysis.
A generated parser is created using a tool called a parser generator (in our case, LALRPOP). Typically, a parser generator requires that the grammar is defined in a Domain Specific Language (DSL), which is then converted into executable code by the generator. In our case, rules were defined in a .lalrpop file, which LALRPOP converted into Rust code.
On the other hand, a hand-written parser involves encoding the parsing rules directly in Rust code, using functions to define the parsing logic for individual nodes.
