Incremental builds in Nix and garnix

We've added incremental compilation to garnix. In this blog, we discuss prior art on incremental compilation in Nix, and describe our own design.

“The ancients had only the books which they themselves wrote, but we have all their books and moreover all those which have been written from the beginning until our time.… Hence we are like a dwarf perched on the shoulders of a giant” — William of Conches, in the earliest known usage of the 'shoulders of giant trope'. Pictured above, a manuscript of a slightly later usage by Bernard of Chartres.

Nix does a great job of caching the work it does, and never doing it twice. If someone you trust built software before (your CI, your coworkers, or the NixOS organization, for instance), you end up for the most part just downloading it instead of building it anew. All of this is very clean, UX-wise: the same commands that would have built things instead first check their caches.

That story, however, only works at the granularity of packages. At the more granular level of modules (or compilation units), Nix doesn't usually do so great. If a single module changes within a package, everything is still built from scratch. For larger projects, such as company monorepos, compilers, or browsers, this can be the difference between compiling in minutes or hours.

Leave a Comment