Company
Date Published
Author
TJ Gibson
Word count
1655
Language
English
Hacker News points
None

Summary

SingleStoreDB's Universal Storage is designed to handle complex queries on large datasets in narrow or wide tables, providing real-time streaming ingest and millisecond response times for web-scale applications. The storage engine's design foundation on speed and complexity ensures app performance, with indexing and query optimization engines working together to guarantee low latency. Key considerations for optimizing schema include shard and sort key selection carefully to maximize query speed, using hash indexes to speed up equality lookups, and adjusting columnstore_segment_rows and Columnstore_flush_bytes to facilitate wider rows loading through memory. SingleStoreDB also addresses traditional pain points of columnstores with the addition of a columngroup index, which improves transactional performance by up to 6x. Query compilation and plancache improve query performance, but may consume additional memory and disk resources. Sizing a wide-table use case requires considering uncompressed data size, compression ratio, CPU:partition ratio, and concurrency requirements, with the ability to scale-out as needed to meet high concurrency queries and large ingest volumes.