Lessons to be Learned from Fastly’s Nightmare Scenario

Internet users the world over experienced massive outages this week as Fastly, one of the largest providers of Point-of-Presence (POP) and Content Delivery Network (CDN) services worldwide, experienced server configuration issues. With the impact of the outage being felt so far and wide, by the tech world and everyday internet users alike, it’s important to take a moment to ask what lessons can be learned in situations like these and what steps we can take to make sure our vital services are safe in times of crisis.

The internet at large erupted into near meltdown at approximately 10:00 UTC on Tuesday morning when reports emerged of massive outages in some of the world’s highest traffic sites and platforms. The source of the outages was soon revealed to be the result of service configuration issues arising within servers at Fastly, a US cloud computing company with annual revenue of over 200 million USD.

Among the sites affected by the configuration issues were e-commerce and payment platforms like PayPal, Stripe, and Shopify, global news outlets CNN, The Guardian, The New York Times, the BBC, and the Financial Times, media platforms including Twitch, Hulu, HBO Max, Vimeo, and Spotify, as well a whole range of other high-traffic global names including Reddit, Amazon, GitHub, Quora, and even the UK’s principal governmental site, gov.uk.

In the fallout – and thankful subsequent resolution – of the mass crash, it’s important to take a quick look at what happened and what we can learn from the current state of how things operate online.

Not too Fast to Criticise Fastly

Before we get too deep into any analysis or commentary on what happened, I think it’s prudent to give credit where it’s due. The teams at Fastly appear to have moved mountains to diagnose and resolve the issue and got an incredible number of things right that another service provider may well not have.

The levels of transparency within Fastly were a credit to the team, who issued regular updates on their awareness and subsequent steps in correcting the issue. Not only were the teams open about the error and proactive in resolving it, the entire issue was cleared up within three hours of it first emerging. Impressive.

What may have started as a disaster for the 10-year-old tech company could well turn into a long-term victory. Before 10am on Tuesday, Fastly wasn’t a name on many lips. A few days on, after having spent the better part of 24 hours trending across Twitter, everybody now knows the name Fastly and, perhaps more importantly, their incredible roster of global clients and lightning-fast recovery turnaround times.

 Looking for Lessons

Despite a potential positive upswing for Fastly, we do need to look for lessons to be learned from its nightmare scenario coming to life. The most poignant takeaway in my view is that we got a real and shocking glimpse into how fragile the internet’s infrastructure can be.

So many of our most trusted and valued sites rely on providers like Fastly for their CDN and POP services. Just look at the list of Fastly’s clients above. When these go down, where are we? If, by some catastrophic turn of events, things had turned out worse, what would the implications be for the global economy? How safe are we relying on a small handful of service providers to keep live and accessible the information and functionalities of our governments, financial transactions, and businesses?

As someone who has worked within the gaming industry for almost 20 years, my instinct is always to relate world events to my industry and question what their implications may be for gaming and the gaming community – both its creators and its players. Thankfully, no gaming developers that we are publicly aware of were impacted by Fastly’s configuration issues. At present, it is unclear whether Fastly has any gaming industry clients. However, some of its competitors (like Amazon CloudFront and Akamai) do.

Many games developers rely on services just like those provided by Fastly to improve download times, prevent DDOS attacks, provide software updates, boost in-game performance for online games, and even create cloud gaming environments for game streaming.

Ask any games developer worth their salt and they’ll tell you just what a disaster an interruption to service like this could mean for them and how many millions of players worldwide could be impacted.

 Finding Independence in Great Solutions

The services provided by companies like Fastly are invaluable. There’s no arguing that fact. Very often, CDN and POP services are vital components of any site or platform’s global reach strategy. But in those unfortunate instances when they lapse, they really make you think about the worst case scenario and how healthy it is to rely on a third party for something as fundamental as having your product, software, or service actually function.

Those that know me will know that this is a bell I ring often. I am a firm believer in the power of cloud technology. But I’m an even greater believer in empowering games creators (and those outside the gaming industry too) with the ability to make sure that technology is proprietary.

With so many games moving online, and online gaming communities thriving in larger numbers than ever before, the real power of cloud technology is in independence. We are fortunate to live in an age where that independence is not only possible, it’s affordable and pretty straight forward.

Launching your own native gaming platforms, with all the advantages of CDN and POP built in, is truly an empowering move for games creators. It helps keep your IP and data safe by keeping everything in-house, it gives you greater autonomy in maintaining and operating your own servers, it cuts costs in the long term (after the initial investment, of course), and it allows you to diagnose, predict, and prevent potential crises far more proactively and transparently than you could do with a third party. And all this is possible while allowing your players to play your games in real time (and on any device!) on your own native cloud software for a consistent and reliable gaming experience.

I believe we should all take a moment to consider and learn from Fastly’s nightmare scenario, reflect on the future, and ask ourselves the question: would I be safer in the long term cutting out the potential for third party liability and simply doing it myself?