Lost in transit: debugging dropped packets from negative header lengths

Debugging networking issues can be challenging, especially when the issue is occurring in the kernel. In this post, we discussed a specific problem where packet segmentation was not working correctly with IPVS's native encapsulation. We outlined our approach to debugging the issue using pwru and kprobe tracepoints to identify where things were going wrong. By analyzing tracepoint output, we discovered that inner MAC header offsets were being calculated incorrectly. We then corrected this in our kernel and observed an immediate improvement in network performance. Finally, we submitted a Linux kernel patch upstream for inclusion in future releases of the kernel.