Fatal Recall? – Understanding the risks associated with the Microsoft copilot + recall feature in Windows

At a recent company event, Microsoft unveiled a promising tool, 'Microsoft Recall.' This feature is designed to streamline the often-frustrating task of locating files, photos, emails, or web pages, eliminating the need to remember specific file locations or browser tabs.

Media outlets have not held back their criticism of the Recall feature, dubbing it a 'privacy nightmare' due to its ability to record user activities on the PC. This has sparked legitimate concerns about the potential for misuse by malicious entities, underscoring the need for heightened vigilance.

Microsoft emphasizes that Recall data remains on the device, is encrypted, and never touches the cloud. It will only be available on PCs that meet specific security standards, including those with the Pluton security chip, and users must opt-in to use it.

Users are empowered with a significant degree of control over what Recall records. They can exclude specific websites or apps and even delete recordings retrospectively. By default, sensitive information from private Edge windows is not recorded, providing a reassuring level of privacy control.

Understanding the inherent risks of data storage, regardless of pre- and post-AI generation, is crucial. This is not a matter of sensationalism but a call to action to comprehend what additional controls are necessary beyond standard data protection measures. We must be prepared to address potential threats that could lead to a data breach, even from a solution designed with the best intentions to enhance productivity.

I found a post from Kevin Beaumont on X today that caught my attention. It discussed the possibility of hacking risks related to this upcoming feature on compatible laptops and desktops running Windows OS.

The Vulnerability Explained

Exfiltration of User Data

Claim: Microsoft claims that hackers cannot remotely exfiltrate Copilot+ Recall activity.

Reality: The feature stores data in an SQLite database, accessible without requiring admin rights. This means that a hacker with minimal access can easily automate the extraction of user data, making it susceptible to breaches.

Ease of Exploitation

Implementation: The SQLite database can be modified with a few lines of code to act as an infostealer. Kevin claims in his post that this has already been demonstrated, indicating how straightforward malicious actors can exploit this feature.

Wide Impact: Messaging apps like WhatsApp, Signal, and Teams are affected. Disappearing, deleted, and sensitive conversations are recorded and stored, rendering privacy features ineffective.

Potential Impacts

Privacy Violations

Personal and sensitive information stored in the Recall database can be accessed and misused, leading to privacy breaches for individual users and businesses alike.

Increased Cybersecurity Threats

By empowering low-level criminals with an easy-to-exploit feature, there is a significant risk of a surge in cyberattacks, targeting the exposed databases across different devices.

Regulatory and Compliance Issues

I hope Microsoft has a team monitoring such vulnerabilities; it is better to take a proactive approach, as regulatory bodies might question their compliance with data protection laws, leading to potential legal and financial repercussions.

Mitigating Risks

I know two well-known banking brands that have not enabled the co-pilot feature on their corporate Office 365 suite for their employees to use. Knowing that some businesses with mature infosec frameworks are treading cautiously is encouraging. When it comes to adopting big-tech AI solutions, the average consumer may be unable to test and isolate the risks associated with using AI solutions offered as a value addition to tools they already use daily.

Microsoft was overdue for a security check, as highlighted earlier in this Wired article earlier this year. The article raised questions about national cybersecurity breaches, increasing the threat levels of non-state actors in other countries.

Satya Nadella later released an internal memo prioritizing security above all other aspects of their work. Despite all the assurances, I advise taking security and privacy protection claims by any big tech firm with a grain of salt today. I also recommend that consumers stay alert for AI-related updates being rolled out for their gadgets and devices. Users and developers can contribute to a safer digital environment by staying informed and proactive.