Why We Started Putting Unpopular Assets in Memory

Cloudflare developed a memory-SSD hybrid storage system that reduces cache hit tail latency and extends the lifespan of SSDs. The system puts unpopular assets in memory to reduce wear on flash memory chips, which are costly to replace. By not caching one-hit-wonders (assets accessed only once), disk writes were reduced by half and corresponding disk hit tail latency was reduced by approximately five percent. A transient cache in memory was implemented, where every asset is missed only once before being promoted to the permanent cache backed by SSDs if popular enough. This approach has been deployed in production, resulting in measurable decreases in cache hit tail latency and a 25% reduction in disk writes at peak usage. Future work includes broader deployment, smarter promotion algorithms, and exploring hybrid storage systems across memory, SSDs, and HDDs.