How to receive a million packets per second
The text discusses the process of achieving 1 million UDP packets per second on a Linux network stack. It begins by explaining the importance of measuring packets per second (pps) rather than bytes per second (Bps), as improving pps is more challenging. The author then describes the experimental setup, including two physical servers with multiple cores and a multi-queue 10G network card. The text goes on to detail various approaches for increasing the number of packets received per second, such as using sendmmsg and recvmmsg syscalls, pinning processes to specific CPUs, and configuring NICs to distribute packets across multiple RX queues. It also discusses the impact of NUMA performance and the use of SO_REUSEPORT flag for load distribution among receiver threads. The author concludes that achieving 1 million UDP packets per second is technically possible on a Linux machine, but it requires careful configuration and optimization to ensure even distribution of traffic across RX queues and receiver processes, as well as sufficient spare CPU capacity. The text also notes that the performance may vary depending on whether the application is processing received packets or not.
Company
Cloudflare
Date published
June 16, 2015
Author(s)
Marek Majkowski
Word count
2129
Language
English
Hacker News points
542