Company
Date Published
Author
Paul Dix
Word count
827
Language
English
Hacker News points
None

Summary

The InfluxDB team has developed a new storage engine called Time Structured Merge Tree (TSM Tree) designed specifically for time series data, aiming to improve disk space usage and query performance. The TSM Tree uses columnar storage and lifts the limitation on the number of fields in a measurement, allowing for efficient updates and compression techniques that adapt to different data types and precision levels. Similarities with LSM Trees are evident, but with improvements such as better compression and reduced disk space usage, resulting in up to 98% less storage space needed compared to previous versions. The new engine is currently available in nightly builds for testing purposes only and requires careful planning to test without disrupting existing data. Extensive testing and community review are encouraged, with detailed write-ups and instructions provided on how to get started.