We’re Ready for You Now: Dragonfly In-Memory DB Now Supports Replication for High Availability

Dragonfly is an in-memory database that can act as a drop-in replacement for Redis, offering high availability and production readiness. Version 1.0 of Dragonfly includes database replication, making it easy to migrate from Redis and suitable for high-availability deployments. High availability refers to a system's ability to operate continuously without downtime or failure, preferably by using built-in failover mechanisms rather than over-provisioning. Dragonfly's replication process is fast and memory-reliable, supporting high throughput, making it perfect for high-availability solutions, especially where existing Redis deployments are struggling. The API provided by Dragonfly is fully compatible with Redis, allowing users to use it with their existing applications and libraries without the need to make any code changes. Database replication is the process of continuously copying the contents of a primary database to replica databases, which are usually running on different servers. This allows systems to avoid a single point of failure — if your primary node fails, a replica can be automatically promoted to be the new master node, becoming the new destination for data writes and the source of any other remaining replicas. Dragonfly uses a shared-nothing architecture that allows data to be replicated in parallel over different connections, with one connection per thread. This shared-nothing architecture is made possible by the fact that Dragonfly was designed to allow multiple threads, making it super fast. Stored data is spread across different shards within Dragonfly, each holding a different set of keys. Dragonfly’s efficient snapshotting algorithm allows it to send update commands to the replica at the same time as sending the full snapshot. Redis, however, sends the full snapshot first — and stores update commands in an in-memory buffer, which can later be sent to the replica (once snapshotting is complete). Dragonfly’s throughput was 7.6 times faster than that of Redis, while its average latency was 7.6 times lower, and its tail (P99) latency was 3.3 times lower. Also, the “full sync” phase of replication was 5.5 times faster in Dragonfly than Redis, and there were no noticeable memory spikes for Dragonfly, unlike Redis. Dragonfly is fully compatible with Redis Sentinel, which can detect when a master instance has failed and automatically promote a replica to be the next master node.