August 30th 2020: Analysis of CenturyLink/Level(3) outage

On August 30, 2020, CenturyLink/Level(3), a major ISP and Internet bandwidth provider, experienced a significant outage that impacted some of Cloudflare's customers as well as other services across the internet. The incident began at 10:03 UTC with an increased number of errors reaching Cloudflare's customers' origin servers. Within seconds, Cloudflare's systems automatically rerouted traffic from CenturyLink/Level(3) to alternative network providers such as Cogent, NTT, GTT, Telia, and Tata. Despite these mitigations, some customers still experienced issues due to their hosting providers only having single-homed connectivity to the internet through CenturyLink/Level(3). The outage appeared to take all of CenturyLink/Level(3)'s network offline, causing a 3.5% drop in global traffic. While the exact cause is not yet known, one plausible scenario involves a bad Flowspec rule that prevented BGP from being announced. It took more than four hours for CenturyLink/Level(3) to resolve the issue.