Full-Text Search with Quickwit and ClickHouse in a Cluster-to-Cluster Context
ClickHouse has recently written a guide on how to use the full text search engine Quickwit with a machine-to-machine configuration. In this blog post, we’ll see how we handle, at Contentsquare, a cluster-to-cluster configuration.
ClickHouse is a column-oriented database management system (DBMS) for online analytical processing of queries (OLAP). Quickwit is a very interesting and promising full text search engine. Initially thought for log management, it can also be used for more than that.
Recently, a developer guide has been introduced by ClickHouse in their official documentation on how to use a Full-text search with ClickHouse and Quickwit. It presents how to use Quickwit with ClickHouse in a simple machine-to-machine context.
However, since both tools can be used in a cluster mode, you might be wondering, “ok but how can I adapt this solution to my cluster and what kind of problems am I going to have?”. And you are right, from a machine-to-machine to a cluster-to-cluster setup it may introduce new complexity and issues. This post will show you how to address them.
This section presents how a query is processed when a ClickHouse cluster works with a Quickwit cluster. In this setup we have on one side, N ClickHouse shards with Distributed Tables. This engine allows distributed query processing on multiple servers, and thus parallelizing reads. On the other side we have M Quickwit nodes.
