Performance doubling with message coalescing

Nagle's algorithm, originally designed for misbehaving applications that write multiple small messages at once, can result in large messages being fragmented and additional delays based on fixed timers instead of network conditions. Disabling Nagle's algorithm is a requirement for some performance-sensitive applications to achieve predictable message latency with fewer outliers. Coalescing network messages at the application level has been introduced as an alternative solution, which can be beneficial in virtualized environments where packet processing throughput may be limited compared to task processing within a VM. Cassandra supports coalescing network messages between nodes and for responses to clients, with different strategies available such as fixed, moving average, time horizon, and disabled. Coalescing message routing has been improved by 8789, which changes how messages are routed between nodes based on size. The impact of coalescing is highly dependent on the configuration and workload, so it's recommended to conduct your own measurements when considering its implementation.