One Billion Row Challenge in Rust

My introductory quotes are getting bigger and more crazy, aren't they? Anyway, in this blog post, we will be doing things a bit differently. It is not one of my usual computer sciency explorations; instead, we will be diving head-first into the One Billion Row Challenge.

A few months back, my friend called me to discuss this challenge, but I did not have the bandwidth at the time, so I am a bit late to the party. This challenge was introduced in January 2024 by Gunnar Morling and lasted an entire month. The original challenge was in Java and involved an innocently simple problem:

That’s it! Gunnar Morling’s blog post has further details on it, including some constraints he put in to make this a bigger challenge. However, in this blog post, we will be doing things a bit differently. It will be a no-holds-barred match with no constraints on language, external dependencies, or hardware. Anything goes!

Let us start by creating one billion temperature measurement values stored in a text file called measurements.txt. These are randomly generated by first building the 1brc project using Apache Maven and then running the ./create_measurements.sh shell script with 1000000000 as its argument. My measurements.txt file is roughly 13GB and looks something like this:

Leave a Comment