Reliably Replicating Data Between PostgreSQL and ClickHouse Part 2 - ClickPipes For PostgreSQL CDC
In part one, we will use PeerDB open source to achieve this using an open source and self managed stack. In part two, we will look at how we can use ClickPipes, the data integration component of ClickHouse Cloud to achieve the same result with a fully managed solution. In part three we include a video walkthrough of both tools for those who prefer a visual demo.
ClickPipes is a component of ClickHouse Cloud which makes it easy to ingest data from sources such as S3, Kafka, and Kinesis into your ClickHouse Cloud Instance.
ClickHouse recently added support for PostgreSQL CDC into ClickPipes, making it possible to easily setup and run a process to replicate data from PostgreSQL into ClickHouse Cloud for both an initial migration and then on an ongoing basis as data is inserted, updated and deleted in the source PostgreSQL instance.
Internally, this solution is built on PeerDB open source. However, rather than using PeerDB to deploy a self managed stack as we described in the previous article, you can use ClickPipes to quickly implement the same solution whilst avoiding the need to setup and operate the process yourself.
