Company
Date Published
April 28, 2024
Author
Richard Artoul
Word count
1947
Language
English
Hacker News points
65

Summary

Tiered storage in Apache Kafka aims to reduce costs by moving historical data to object storage, but it often ends up being penny-wise and pound-foolish. It can make Kafka more difficult and complicated to deal with, rather than simpler, and introduces new operational problems and failure modes. While it may reduce some costs, such as cloud networking fees, these savings are often offset by increased complexity and unpredictability. In practice, tiered storage does not live up to its promises of simplifying operations or making Kafka more efficient, and many users have been disappointed with the results. Instead, some vendors are rebranding existing products as "tiered storage" without actually implementing any significant changes.