CrowdStrike or the resilience conundrum

We all woke up this morning to news of a major computer outage all over the world affecting large businesses such as airlines, hospitals, railway companies, telecom operators, and, according to NBC News, some 911 emergency services in the US.

Game designer Robin D. Laws just posted this on Mastodon, which I find funny because there's a strong kernel of truth to it:

"This is how the world ends, not with a bang, but with a Windows 365 cycling reboot error."

What's going on seems to be a little bit more complex (and hopefully a little less dire) than that, but is indeed an issue (from what I understand) driven by a CrowdStrike update initiated last night, but impacting Windows OS and effectively decommissioning computers that belong to networks that use CrowdStrike for their security until either the solution is disabled or the patch is installed. Apparently this needs to be done on a computer by computer basis.

The global impact of this update highlights once again a conundrum of modern network security: as attacks become more and more sophisticated, it is virtually impossible for businesses to protect themselves without relying on a small number of security software providers such as CrowdStrike. Paradoxically, this creates a single point of failure of sorts as a huge proportion of businesses worldwide use the same solutions. If these solutions fail, a large proportion of businesses go down, and cascading effects impact others due to interdependencies (a number of unaffected airports are closing down because the airlines or other airports are...)

At a time when the European Commission puts a strong emphasis on resilience and cybersecurity through things like Articles 40 and 41 of the EECC as well as directives such as NIS2, this raises interesting and complicated questions about the role of policy in this field. A lot of these EC mandates have yet to be implemented by national regulators, and we at Plum Consulting have assisted regulators in figuring out how to implement these necessary changes.

The incidents from today however, raise a real question about the role and limitations of policy when it comes to resilience and cybersecurity: if imposing measures on various critical businesses leads to an increasing dependency of said businesses on the same providers to meet the obligations put upon them, this in itself is a new and systemic risk of failure should said providers themselves crash or fail.

This is, in another guise, the tension between centralised and highly protected architectures, which rarely fail, but have massive impacts when they do, and decentralised but potentially weaker architectures, which may individually have higher risks of failure, but much lower impact when they fail.

There may be a risk that policy interventions designed to enhance security through standardised requirements lead to reliance on just a small number of vendors who have the capability to meet these standards at scale. Paradoxically this could lead to overreliance on systems which are themselves capable of failure. Maybe some thoughts need to go into different, less centralised models that would mitigate the systemic risks a little better?