My internship: Brotli compression using a reduced dictionary

Brotli is a state-of-the-art lossless compression format supported by all major browsers, offering better compression ratios than gzip. One of its key features is the built-in static dictionary that contains various strings in multiple languages and supports multiple transformations to increase versatility. The open-source brotli library has 11 predefined quality levels for the encoder, with higher levels demanding more CPU for better compression ratios. The Cloudflare team improved on the limited use of the static dictionary approach by adding optimizations that enhance compression at levels 5 through 9 without significantly impacting performance. They achieved this by using a larger, specialized subset of the dictionary and employing heuristics to improve the compression ratio. The resulting reduced dictionary is generated based on statistics about matches in representative content data sets for HTML, CSS, and JavaScript files. The improved dictionary approach allows for better compression of web content without requiring higher CPU usage compared to using a higher compression level. This enhancement benefits small files more significantly than large ones, making it an effective solution for optimizing web traffic.