Why We Started Putting Unpopular Assets in Memory
Cloudflare developed a memory-SSD hybrid storage system that reduces cache hit tail latency and extends the lifespan of SSDs. The system puts unpopular assets in memory to reduce wear on flash memory chips, which are costly to replace. By not caching one-hit-wonders (assets accessed only once), disk writes were reduced by half and corresponding disk hit tail latency was reduced by approximately five percent. A transient cache in memory was implemented, where every asset is missed only once before being promoted to the permanent cache backed by SSDs if popular enough. This approach has been deployed in production, resulting in measurable decreases in cache hit tail latency and a 25% reduction in disk writes at peak usage. Future work includes broader deployment, smarter promotion algorithms, and exploring hybrid storage systems across memory, SSDs, and HDDs.
Company
Cloudflare
Date published
March 24, 2020
Author(s)
Yuchen Wu
Word count
2159
Language
English
Hacker News points
16