Apache Pulsar is an open-source, cloud-native distributed tiered messaging system that is a part of the Apache Software Foundation. This distributed m

Apache Pulsar Tiered Storage with Storj DCS

submited by

Style Pass

2021-05-20 18:00:06

Apache Pulsar is an open-source, cloud-native distributed tiered messaging system that is a part of the Apache Software Foundation. This distributed messaging and streaming platform manages hundreds of billions of events per day and is widely deployed across enterprise-grade systems, including Splunk, Overstock.com, Verizon, Comcast, and Toast.

Pulsar features Tiered Storage that allows developers to offload the non-compacted data, making it economical to store for long periods of time.

Today, we would like to showcase an open-source integration and walk through the process of offloading event data from Pulsar to Storj DCS (Decentralized Cloud Storage).

"Pulsar has reshaped the way that the industry thinks about a modern messaging and event streaming architecture," said Chris Latimer, VP of Product, Streaming at Datastax. "At the same time Storj's Decentralized Cloud Storage has pushed the boundaries of distributed persistence. When you combine these technologies you end up with a highly efficient solution both in terms of cost and performance."

Tiered Storage in Apache Pulsar solves this problem, making it easier to reduce the total cost of Data Ownership related to messaging systems, while still guaranteeing the integrity and availability of the data.