In 2023, I wrote about how we’ve tuned Ruby’s garbage collector for Shopify’s monolith, including how we implemented out-of-band garbage collect

Next Generation Out of Band Garbage Collection

submited by
Style Pass
2024-10-31 17:30:08

In 2023, I wrote about how we’ve tuned Ruby’s garbage collector for Shopify’s monolith, including how we implemented out-of-band garbage collection to reduce the impact of major collection on latency.

While the latency improvements were massive, we weren’t entirely satisfied with the heuristics used to trigger out-of-band garbage collection. It was purely based on averages, so we had to trade latency for capacity. More importantly, it didn’t fully eliminate major collection from request cycles, it only made it very rare.

In March 2024, during our annual Ruby Infrastructure team gathering, we fleshed out the details of the new feature we wanted, and Matthew Valentine-House started working on a proof of concept, which we then deployed to a small percentage of our production servers to see how effective it could be.

First, we needed a way to entirely prevent the Garbage Collector from automatically performing a major collection, but also to stop promoting objects to the old generation. Ideally in a web application, aside from some in-memory caches, no object allocated as part of a request should survive longer than the request itself. Any object that does is probably something that should be eagerly loaded during boot, or some state that is leaking between requests. As such, any object promoted to the old generation during a request cycle is very unlikely to be immortal, so promoting it is wasteful.

Leave a Comment