Benchmarking Quickwit vs. Loki

In 2019, Grafana launched Loki, a new log aggregation system, to tackle the challenges commonly faced by teams operating and scaling Elasticsearch:

The consensus was that Elasticsearch wasn't the right solution for efficient log management. Loki aimed to provide a cost-efficient through a minimalistic indexing approach that indexes only metadata (labels), inspired by Prometheus’s labeling system, and leveraged cheap object storage. Humio1 had a similar approach with its unique index-free architecture.

In 2021, we launched Quickwit with the same goal but a different approach. While we also targeted cost-efficiency and use object storage, we maintained the inverted index, combining it with a columnar storage. Our ultimate goal was to provide a search engine on object storage, retaining the strengths of traditional search engines while addressing the good complaints coming from Elasticsearch users.

Today, as Quickwit is gaining traction and being adopted by companies at petabyte scale2, we are often asked about the trade-offs of using Quickwit vs. Loki for log management.

Leave a Comment