Optimizing global message transit latency: a journey through TCP configuration

Ably provides realtime messaging services with low latency. A new customer experienced unusually high variance in message latency between Europe and South East Asia. Initial investigation revealed that the issue was not due to processing delays but propagation delays on the TCP/IP connection. The problem was traced back to the tcp_slow_start_after_idle setting in the Linux TCP stack, which resets the congestion window after an idle period. Disabling this setting within Docker containers eliminated the latency tails and improved message delivery times for the customer's bursty traffic. This highlights the importance of understanding and optimizing network stack configurations tailored to specific use cases.