Internet company CloudFlare has published details about the extensive internet problems that Europeans experienced on Monday. The content delivery network blames Telia and calls the outage unacceptable. The cause would have been human error.
On Monday, many European internet users experienced connectivity problems with multiple services, such as Slack, WhatsApp and Reddit. Amazon AWS and CloudFlare were also affected, but the blame turned out to be with the Swedish Telia. Telia Carrier is one of the tier 1 network providers and manages an extensive fiber optic network, with which the provider is responsible for an important part of the internet backbone. CloudFlare is a content delivery network, or cdn, and must offer fast and stable connections thanks to its many connections to internet exchanges.
CloudFlare detected a significant packet loss at Telia Carrier last Friday, but the company quickly resolved that problem. Monday it happened again and this time the problems were more extensive, according to a graphic overview that CloudFlare has prepared. “As transit providers are normally very reliable, they usually resolve their issues quickly. In this case, that didn’t happen and we had to close our gates with Telia,” the company writes.
Packet loss on June 20, measured by CloudFlare
The cdn registered large numbers of 522-http errors, indicating that its servers could no longer reach its customers’ servers. CloudFlare tried to redirect traffic to other, in the company’s words, “higher-performing providers,” but failed to prevent the issues from impacting customers and web users.
CloudFlare uses the border gateway protocol to route the data packets. “Bgp works well to keep interconnections stable, but it doesn’t have a mechanism to detect packet loss and performance issues.” The company itself says it is working on a mechanism to actively register packet loss and redirect traffic, but this is only being used in remote and small locations. Over the next two weeks, CloudFlare would expand this technology to more locations.
The Director of CloudFlare calls Telia’s reliability over the past sixty days is ‘unacceptable’. His company has reduced its dependency on Telia’s network until the issues are resolved. Telia has not yet responded. According to The Register, it was a human error, where someone misconfigured a router. The site claims that when configuring a router, a technician confused Hong Kong and Europe, causing it to go wrong. It is not clear where the site got that information from and Telia has not confirmed the information.