Large WebRTC Video Grids: Managing CPU and Network Constraints

Senior Architect Ben Weekes at Agora has developed a solution for displaying large video call grids in a browser environment, which was previously believed to be impossible due to varying CPU/GPU capacity across devices. The technology could be useful for virtual events and company meetings where the energy of a large group contributes to the experience. The main challenges include processor load, lack of hardware information from browsers, and changing network conditions. To tackle these issues, Weekes ensures that only necessary pixels are delivered, monitors volatility in render frame rates and outgoing frame rates, and stays ahead of changing network conditions by monitoring real-time estimations of available downlink bandwidth and NACK rate. The open-source algorithm for this large WebRTC grid experiment is available on GitHub through Agora's Multichannel SDK.