Bring your own REST APIs for Apache Kafka

submited by

Style Pass

2022-06-22 21:00:05

The Apache Kafka open source message streaming platform is widely adopted as the core of many modern event-driven application architectures.

Use cases include: publish and subscribe messaging, website activity tracking, log or metrics aggregation, stream processing, and more.

Although languages running directly on the Java VM, such as Java and Scala, have fully supported client libraries provided by the Apache Kafka project, other language client libraries are maintained separately by independent projects in the Kafka ecosystem.

The Kafka client APIs and network protocol are complex, so it is challenging for other languages to reach the same level of maturity, even for the baseline capabilities. As new features and functionality are added to the official client libraries, the other languages often do not keep up with the same pace of development.

These challenges have given rise to projects such as the Confluent REST Proxy, Karapace REST Proxy and Nakadi Event Broker. In each case, a system level REST API is provided by the project, allowing clients to interact with Kafka topics using standard HTTP clients and without the need for a native Kafka client library.