How to keep calm and debug smart TVs

A Mux customer reported inconsistent freezing issues on Roku devices while playing videos. The problem was not reproducible consistently and only occurred on Roku devices, making debugging challenging. After extensive testing across multiple Roku devices and handcrafting HLS manifests, the team discovered that the decoderStats object indicated errors in the video stream itself. They found a pattern where the freezing cycles matched the chunk length of the input stream. To reproduce the issue, they created a sample app with access to the decoderStats object and observed that something was wrong with the individual segment files themselves. They used ffprobe to examine the .ts files but could not find any significant issues. The team purchased older Roku models on eBay to test different versions of the video asset, but they still couldn't reproduce the issue consistently. Upon closer examination of CDN delivery logs, they noticed that the client requested a partial segment and never requested the rest, which lined up with the freezing issue. They repurposed the static case from earlier to test this problem by forcing partial responses for segments and observed the issue. The team discovered that Mux's Content-Range response headers were causing confusion for Roku clients when faced with an unknown content length. As a fix, they decided not to serve range requests for these User Agents, which resolved the issue without any impact on Quality of Experience. This case highlights the importance of collaboration between different teams and the persistence required in solving complex bugs.