Deserializing Binary Data Files in Rust

submited by

Style Pass

2021-06-20 05:30:06

The other day, someone on the Rust user forums posted a question that really nerd-sniped me. They had data generated by a C++ program and were wanting to load it into a Rust program, but when asked what format the data was in the author didn’t provide some something like a JSON schema or Protobuf file, instead they just got the definition for a C struct.

A common method for “serializing” data in C is to create a struct and directly write its bytes into a file, the “deserializing” is just a case of reading the data out of the file and interpreting it as your type. This technique is actually kinda genius when you think about it, it makes no intermediate copies or heap allocations because the OS’s read() function will literally write your data to its destination, and there are no extra dependencies or complicated serialization frameworks involved.

I’m not going to go into this technique’s drawbacks (of which there are many… there is a reason we use things like JSON and Protocol Buffers nowadays, after all) and instead let’s just focus on how to read these sorts of files.