How Waiting Room makes queueing decisions on Cloudflare's highly distributed network

Cloudflare's Waiting Room solution uses a sophisticated approach to queue users attempting to access a website when there is high traffic or potential overload on servers, preventing crashes and downtime. The system divides available slots among the company's workers across data centers worldwide based on real-time utilization levels. Initially, they evenly divided these slots; however, this led to some queuing happening too early at a data center and potentially exceeding customer set limits during sudden spikes in traffic after periods of low utilization. To address these issues, Cloudflare implemented "counters" as part of the Waiting Room system that help manage synchronization between workers and ensure that decisions about whether to grant access or queue users are made with minimal latency at the right time. Counters divide available slots among data centers based on real-time traffic patterns, avoiding early queuing while still respecting customer set limits. This approach has been successful in maintaining website performance without compromising user experience due to negligible added latency for new users trying to access a site during high traffic periods. Cloudflare continues to refine the Waiting Room system by learning from different types of traffic patterns and adapting their approach accordingly, ensuring optimal protection for customer websites.