How to build a scalable DNS load balancing system | Algolia

Algolia initially decided against using load balancing infrastructure between users and their search API servers to keep things simple, remove potential single points of failure, and avoid costs. Instead, they relied on round-robin DNS for load distribution. However, this approach was suboptimal when a large customer with a back-end implementation experienced heavy search loads during Black Friday, leading to slow and failing queries. To address this issue, Algolia implemented a rudimentary Nginx-based load balancer as a workaround. They later developed a more scalable and maintainable solution using OpenResty and Redis for caching, with lb-helper handling dynamic data. Currently, they still primarily rely on DNS-based load balancing but are aware of its limitations in certain situations. The experiments have also led to the development of an abstraction layer on top of their search infrastructure, making failures, changes, and scaling more transparent for customers.