The more centralised the internet becomes, the more susceptible it is to catastrophic failures.
Where were you when the internet went down? Last Tuesday, late in the evening, I was clicking on a Times of India link and I got hit by an “Error 503 Service Unavailable” message. I thought perhaps they had shadily taken down the story.
Of course, then I went on Twitter and clicked on something else. Error 503 yet again. I also noticed that emojis were not loading on Twitter; instead, their text descriptions would load vertically, giving the whole timeline a surreal appearance. It felt like the setting for a creepy horror movie, except online.
On WhatsApp and Twitter, I discovered it wasn’t me, it was the internet. Everyone was complaining. While some bits of the internet (like Twitter minus the emojis) were still working, vast swathes of it were simply not loading for many people. Things mostly resolved themselves in about an hour, but it was only some time later that everyone found out that the problem was with Fastly, a content delivery network.
When the internet works properly, you take it for granted. It’s like oxygen. You don’t really think about it, or the process behind the scenes. Typically, the disruptions you face are local: your router is acting up, or you forgot to pay your broadband bill, or your cellphone’s data connection is patchy. You’re resigned to these disruptions because they are familiar, proximate and, to a certain extent, within your control.
But beyond that last mile lies the vast and complex skeleton of the internet, a vital part of which is a content delivery network, or CDN, like Fastly. Essentially the internet is all about serving up content and interactions (whether it’s text, images, audio, video, emojis, etc) from a server to a device belonging to you, the user. But there are many points of failure in this incredible labyrinth, as has been evidenced in multiple outages over the past few years. While it’s impossible to describe the entire skeleton of the internet at anything short of book length, we’ll take a quick look at a few of its elements and the ways in which chunks of the internet can go down if one of them malfunctions.
In the simplest, most basic form of internet access, a static page of content can be hosted on a server. This server could just be a computer sitting in someone’s house or, more typically, a server sitting in a data centre somewhere, and that page is delivered over a network to your computer. As this process expands – in terms of nature of service, geographical range, volume of content, complexity of content, and number of operators involved – this process gets infinitely more complicated and sprawling.
These days, the CDN is probably the first thing you unknowingly encounter as you surf the internet. CDNs are networks of servers around the world, which distribute duplicated versions of many elements of a website so that they can be delivered quicker to the end user by being physically closer to them.
For instance, when you open a video on Netflix (which uses its own CDN), Netflix would have stored multiple copies of that video in servers in different parts of the world, depending on its expected demand for that particular video. So, a user clicking on the “Play” button will be served up the video from its nearest server in the quickest possible timeframe with the lowest possible latency.
For a video in an Indian language, Netflix’s CDN may prioritise storing it in servers in or closer to India and places with significant Indian populations, so that the bytes comprising the video find the shortest or quickest route to your laptop or smartphone. Now, when a CDN like Fastly goes down, all the websites that use it to deliver their content to you will also go down. And, as we discovered recently, large portions of the internet depend on Fastly.
Another critical category of service powering the internet is that of the cloud service provider. Cloud service providers offer a vast array of services, including cloud computing, cloud storage, database, analytics and pretty much everything you would need to deliver a website, app, product or service over the internet.
Many of you may remember the Amazon web services, or AWS, outage of November 2020 that took out many websites and apps. AWS and other cloud service providers like Microsoft Azure and Google Cloud are behind a rapidly increasing portion of the internet these days. These service providers also work by having a network of data centres and server farms all over the world. For instance, the major cloud service providers have servers in Delhi, Mumbai, Chennai, Bengaluru and Hyderabad. (See map for more.)
With so much of the internet depending on one of three big vendors, an outage with any of them – even if it’s relatively small and localised, like the AWS outage – can have a debilitating impact on internet access for millions of people.
Now, apart from service providers, the physical infrastructure of the internet is also a vast network, comprising data centres, internet exchange points, and submarine cables, among other things.
Submarine cables have been the backbone of international communications right from the 19th century, when they carried telegraph messages, coded using codes like the Morse code, across the world. Right now, a vast network of submarine cables connects the whole world to the internet. India has multiple cables landing in seaside cities like Mumbai, Chennai, Kochi, Thiruvananthapuram and Thoothukudi (see map). Faults with submarine cables have caused numerous localised internet outages, the most memorable of them being perhaps the 2008 submarine cable fault, where three major submarine cables, especially those connecting India to the internet, suffered damage, leading to a serious outage.
Apart from the above, there are many other critical elements of the internet, including internet exchange points, where different internet service providers and content delivery networks connect to each other, that can also fail, leading to some degree of loss of functionality for you, as a user, even if it’s fleeting.
The beauty of the internet is that it is distributed. There are myriad ways for a packet of data to get from point A to point B. This essentially decentralised aspect of the internet has been a critical element in its impact, growth and resilience.
With this background, it’s a concerning aspect that much of the backbone of the internet is getting consolidated, whether it is with cloud service providers like AWS and Azure, or CDNs like Fastly. The more centralised the internet becomes, the less resilient it is and more susceptible to catastrophic failures, whether caused by internal faults or external attacks. The Fastly outage is the latest reminder that this is a trend that needs to be halted, if not reversed, for the sake of the internet’s future.
Contact the author on Twitter @vinayaravind.