CrowdStrike: The Case Study

Curator: Abhishek Kumar

Author: Jasmine Dhamija Parneet Kaur Abhishek Kumar

Banner by: Yugen Sevda

CrowdStrike

A routine cybersecurity update intended to bolster defenses instead triggered global chaos when CrowdStrike, a key Microsoft partner in cybersecurity, rolled out an update that inadvertently crippled systems worldwide. This update led to widespread disruptions, affecting crucial services from airlines to hospitals and even impacting everyday operations like ATM transactions. The fallout was particularly severe for the aviation industry, where major airlines like Delta Airlines faced significant operational hurdles due to system failures and Blue Screens of Death, commonly being referred to as BSOD. Reports indicate that nearly 85 million Windows devices were affected.

In response to this crisis, the U.S. House Committee demanded answers, summoning CrowdStrike CEO George Kurtz to testify about the outage's origins and implications.?

“The incident underscored broader vulnerabilities in our interconnected digital infrastructure, prompting urgent discussions about the reliability of cybersecurity systems designed to protect but capable of inadvertently disrupting global operations.”

-George Kurtz, CEO of CrowdStrike

CrowdStrike is an endpoint protection platform that claims to protect 298 of the Fortune 500, 538 of the Fortune 1000, and 43 of the 50 U.S. states. This incident highlights the potential scale of such havoc, emphasizing the delicate balance between security enhancement and operational stability in our increasingly digital world.

Here's What Happened

On July 19, 2024, CrowdStrike rolled out an exciting update to their Rapid Response Content for the Windows Falcon sensor, this update integrated the motion detection feature into the OS that uses machine learning to detect security threats.?

CrowdStrike’s ‘The Rapid Response Content’ update is delivered to the Falcon sensor using a dynamic content update mechanism. The contents of the update are first validated by a Content Configuration System in the cloud and then deployed to the sensor. The sensor's Content Interpreter then reads and interprets the Rapid Response Content, enabling the Sensor Detection Engine to observe, detect, or prevent malicious activity.

However, a bug in this content validator caused faulty content to be deployed, resulting in system crashes and the "Blue Screen of Death" for Windows hosts running sensor version 7.11 and above between 04:09 UTC and 05:27 UTC.

This incident stemmed from the deployment of two additional IPC Template Instances, where one instance containing problematic content passed validation. When this faulty content was loaded into the Content Interpreter, it triggered an out-of-bounds memory read, causing the crashes.

Case Studies: Sector-Specific Global Disruptions

This global halt brought substantial challenges to various sectors, affecting everything from travel to healthcare. Each domain encountered significant disruptions in its operations. The following discussion explores some of the distress experienced around the world:

Healthcare

We had to witness multiple hospitals in Europe reporting cancellation of procedures and closure of clinics. As per the reports of Bloomberg, Reuters, and posts on X by medical officials, the National Health Service systems in the UK were widely affected, the booking systems used by doctors went offline, as doctors were unable to access blood tests, patient histories, and scans. Similarly, in the US, the outages affected call centers, patient portals, and other operations in the healthcare centers. In addition, 911 and other emergency systems went down in major metropolitan areas of New York and New Hampshire.?

Banks, Finance, and Corporations?

One can only imagine the reliance of modern enterprises on the services provided by CrowdStrike; this incident didn’t merely lead to a seizure of company operations but also major revenue losses. CrowdStrike itself saw its shares plunge on Friday, where they opened down more than 14% and closed down about 11%.?

The London Stock Exchange Group's (LSEG) news website RNS, the platform that is used by companies to distribute their price-sensitive regulatory announcements was also disrupted due to this outage. Besides the UK, the offices of international bankers like?Bank of America, JPMorgan Chase, and Nomura Holdings had to resort to backup systems as employees were unable to log into their systems.?

This major media outage also affected several high-profile companies like FedEx and Meta. FedEx had to experience disruptions in shipments, while the content moderators on Meta's Facebook faced difficulties. Additionally, American Express encountered temporary issues processing transactions.

The situation at Tesla required CEO Elon Musk to address the issue on his social media platform X (formerly known as Twitter). Musk stated that Tesla had "deleted" CrowdStrike from all its systems. In a previous post, he remarked that the outage had "given a seizure to the automotive supply chain."

Media?

The media outage caused by this incident impacted major broadcasters around the world, the world was shaken as some news anchors went on-air from dark offices, with computers showing blue error screens.

In Britain, Sky News went off-air, while in the US, TV stations like KSHB-TV in Kansas City, Missouri, aired Scripps News instead of local news until around 5:35 a.m. Despite similar issues, Scripps managed to resume local broadcasts at 90% of their stations by 12:00 UTC, showing resilience. Even Australian national outlets like ABC and Sky News Australia couldn't broadcast for hours. The incident revealed just how much media outlets rely on technology, almost like a lifeline, to keep their operations running smoothly.

Airlines and Travel?

Airline services across the world using Microsoft Windows services were brought to a grinding halt as the Blue Screen of Death dawned on their screens.

Airports from Berlin to Delhi, and Los Angeles to Singapore, struggled with delays and cancellations, leaving passengers stranded during their busy travel schedules. The US Federal Aviation Administration (FAA) reported significant impacts on United Airlines and Delta Air Lines, while American Airlines and Spirit Airlines also had to halt flights temporarily.

At the Rajiv Gandhi International Airport in Hyderabad, staff had to resort to issuing handwritten boarding passes to manage the delays. Times of India described this situation as a return “to the stone age.”

How did it Remediate?

On July 19, 2024 (05:27 UTC), CrowdStrike released a set of guidelines for reverting this world-halting issue.?

According to the guidelines, users encountering the Blue Screen of Death (BSOD) should follow the provided solution to regain control of their devices:?

• Boot your computer into safe mode and log in with your credentials.
• Navigate to task manager, run a new task, and launch PowerShell for troubleshooting.
• Navigate to the drivers and CrowdStrike directory, locate and delete the file named “C-00000291*.sys” using the 'RM -Force' command while utilizing PowerShell for more flexible navigation and autocompletion.
• Reboot your system normally and you'll be good to go.

Conclusion

This CrowdStrike incident stands as a stark testament to the complexities and perils of modern cybersecurity. Intended to bolster defenses, the update instead unleashed unprecedented global chaos, emphasizing the critical need for rigorous validation and fail-safes in software deployment.

The sweeping impact across vital sectors like healthcare, finance, media, and travel underscores the profound interconnectedness of our digital infrastructure and the potential for far-reaching disruptions. This incident has sparked a clarion call for cybersecurity firms and their clients to prioritize robust testing protocols, continuous monitoring, and rapid response strategies.

Ensuring a balance between security enhancement and operational stability is not just a necessity but an imperative in our increasingly digital world, where the stakes have never been higher.

Read more about the incident here!

AI Weekly is a collective effort by EvolveAI, to inform our followers of the constantly evolving world of AI.

Follow us on Instagram to know more about EvolveAI and the events we conduct on a regular basis.

Thank You for reading the fifth edition of AI Weekly, see you next week!