Building a better and scalable system for data migrations
Recently I read about a new Rust web framework called rwf. I was especially curious about its approach to handling database migrations, as many frameworks released in recent years tend to either overlook this or provide a mechanism that won't scale to larger projects. Reading through the documentation I learned that rwf falls in the second category: it generates two SQL files, one for migrating forward and one for migrating backwards, and evaluates the appropriate files based on the migration direction.
The problem here isn't necessarily the choice of using SQL (though it certainly doesn't help), but rather that the system is built in such a way that it becomes useless the moment your team and/or database starts growing in size.
I've been thinking about what a better solution to data migrations might look like for a while, so let's explore my current thoughts on the subject.
Since developers love to argue about terminology rather than focusing on the subject as a whole, we'll start off with defining a few terms so it's clear what we're talking about.
