Infinite Storage in Confluent Platform

Confluent has introduced Tiered Storage in its Confluent Platform 5.4, allowing operators to add an additional storage tier for data. The feature aims to make the platform more scalable, cost-efficient, and easier to run operationally. Tiered Storage separates the concerns of data storage from data processing, enabling independent scaling of each. By offloading the majority of data to a remote store, it reduces the time and cost required for rebalancing, expanding or shrinking clusters, or replacing failed brokers. This also allows users to use different storage types at the broker level to improve performance and cost profile. Tiered Storage is currently in its preview phase as a feature of Confluent Platform and supports Amazon S3.