Every request, every microsecond: scalable machine learning at Cloudflare

In this article, we discussed how Cloudflare developed a new design for its Bot Management platform using Rust and machine learning features. We explored the various challenges faced by their previous system and delved into the innovative solutions they implemented to overcome these hurdles. Key highlights of the redesign include: - The introduction of mmap-sync, a newly developed Rust package that provides efficient memory-mapped file operations with wait-free synchronization. - The development of two new components - the bliss service and library - designed for optimal batch processing of vast data quantities and minimal resource usage, respectively. The system redesign brought about significant improvements in latency and machine learning feature availability, as well as enhanced efficiency in resource utilization. Overall, this upgrade has led to a more robust and performant Bot Management platform at Cloudflare.