Simplify migrating from Kafka to Pulsar with Kafka Connect Support
The transition from Kafka to Apache Pulsar can be complex and risky, often resulting in organizations being locked into suboptimal situations. However, mitigating these risks is possible by making the transition process iterative and breaking it down into manageable steps. The ecosystem supporting this transition includes built-in connectors for pulling/pushing data between Pulsar and Kafka topics, as well as the Kafka Connect Adaptor (KCA) Sink and Source. Apache Pulsar 2.8 offers improvements to the Java Pulsar Schema API and the Pulsar IO API that simplify schema handling in Sinks. These enhancements make it easier for users to develop Pulsar IO Sinks without being tied to a particular schema at build time, and support KeyValue messages seamlessly. The Kafka Connect Adaptor completes the compatibility ecosystem between Kafka and Pulsar, allowing an iterative transition from one system to another while supporting native Kafka clients with Pulsar and data transfer between two systems.
Company
DataStax
Date published
June 17, 2021
Author(s)
Enrico Olivelli
Word count
1208
Hacker News points
None found.
Language
English