Solid Queue v1.0 released
We’ve just released Solid Queue v1.0.0, right before speaking about it at Rails World. This version has come a long way since we published the first version, 0.1.1, back in December 2023, with 132 merged pull requests and 126 closed issues, and the help of multiple contributors.
More importantly, we’ve completely migrated HEY, our email and calendar service, over from Resque. In fact, HEY Calendar was launched in January directly using Solid Queue for all jobs. Some issues we had after that launch is what drove batch operations support. We’ve also started moving jobs in our biggest app, Basecamp 4.
Currently in HEY we’re processing about 20 million jobs per day, using 800 workers, 4 dispatchers and 2 schedulers, spread over 74 VMs running in two datacenters, that we deploy using Kamal. Some of the queues are quite overprovisioned from our past Resque setup; we’ll reduce the number of workers, VMs and servers in the future, and reassign them to other work. These 74 VMs are sized differently depending on load and priority, with 2 to 6 cores (we have AMD EPYC 9454 48-Core processors) and 4G to 27G memory allocated for each of them. The physical servers where these live are running a mix of VMs for web, jobs and other apps, we don’t run dedicated machines only for HEY.
We’re running Solid Queue in a separate database from the main app. We also have the tables for Active Storage and Action Text in separate DBs. We run a primary and two replicas for each, and the four databases share the same physical servers. For the Solid Queue database, we’ve allocated 32 CPUs, 64G memory and 350G disk.
