Events, Event Sourcing, and the Path Forward
Distributed systems pose all kinds of challenges. And we’ve built them in the web age, when the tech of the wider Internet is what we use in microcosm to build the underpinnings of our own systems. Our industry has done somersaults to try to make these systems work well with synchronous calls built on top of HTTP. This is working at scale for a number of companies and that’s just fine. But if you were to start from scratch is that what you would build? A few years ago, we had that opportunity and we decided, no, that’s not what we would build. Instead, we built a distributed, asynchronous system centered around an event bus, and it has been one of the best decisions we’ve made. Most of the things that make a service architecture painful are either completely alleviated or mostly so, with only a few tradeoffs. Here’s what we’ve built at Community, and some of what we have learned.
The advice du jour is that you should start with a monolith because it allows you to iterate quickly, change big things more easily, and avoid any serious consistency issues—assuming you back your service with an ACID-compliant DB. That’s good advice that I also give to people, and we did that. That scales pretty well, but becomes an issue as you grow the team and need to iterate on independent work streams.
