Structured Editing and Incremental Parsing

submited by

Style Pass

2024-11-27 13:00:08

As someone who has from time to time written on parsing, and even published some parsing research, I get asked questions about parsing fairly regularly, some of which I’m even capable of answering.

One thing that I’ve increasingly been made aware of is a widening interest in “fancier” ways of editing text. In essence, most people who’ve been programming for a while come to realise that having an editor that only thinks of a program as a sequence of UTF-8 characters is rather limiting. I suspect the rapid proliferation of the LSP (Language Server Protocol) has opened even more people’s eyes to what the editing of programs could be.

One long-standing approach to better editing is structured editing (sometimes called “projectional editing”). The basic idea is to have an editor which fully understands the syntactic structure of the language you’re editing. This has various benefits: the editor can give instant feedback about what the next thing the user can type is; it makes semantic-based feedback (e.g. about the static typing of a program) much easier; it enables things like “select this function so I can copy it” 100% accurate, instead of using slightly dodgy heuristics; and so on.

To the best of my knowledge, the first structural editors were for Lisp dialects in the late 1960s. The best modern structured editor I’ve used is JetBrains’ MPS. If you haven’t tried using a structured editor, I’d encourage you to do so: it’s a much different experience than using even an LSP-powered editor and can change how you view the editing of programs.