Part 3: Log file compression with Zstandard VFS in SQLite + benchmark
Part 1: Log file compression with Gzip and Zstandard + benchmarkThis is the first part of a series of articles diving into log files compression techniques.Part 2: Log file compression with SQLite + benchmarkThe second part utilizes SQLite as an embeddable storage that not only provides querying capaibilites but also is able to compress the logs itself.(currently reading) Part 3: Log file compression with Zstandard VFS in SQLite + benchmark The third part dives deeper into compression capabilities of SQLite using Zstandard compression algorithm.(work in progress) Part 4: Log file compression with Zstandard selective column in SQLite The fourth part explores database level compression of specific columns within SQLite.
The second part utilizes SQLite as an embeddable storage that not only provides querying capaibilites but also is able to compress the logs itself.
I started looking into existing solutions to compress SQLite database while retaining querying capabilities. One very perspective candidate was https://github.com/jtarchie/sqlitezstd repo which is described as:
