Mitigating a token-length side-channel attack in our AI products

Researchers from Ben Gurion University discovered a novel side-channel attack that can be used to read encrypted responses from AI assistants over the web. The attack involves intercepting the stream of a chat session with an LLM provider, using network packet headers to infer the length of each token, extracting and segmenting their sequence, and then using dedicated LLMs to infer the response. To mitigate this vulnerability, Cloudflare added padding to token responses with random length noise to obscure the length of tokens in the stream, thereby complicating attempts to infer information based solely on network packet size. This protection is now automatically applied to all users of Workers AI and AI Gateway.