Navigating the Ban on Microsoft Co-Pilot in the US Government: Implications and Actions for Security Professionals

Why Should I Care?

In an unprecedented move, the US government has banned the use of Microsoft Co-Pilot across its departments, citing significant security concerns with the sharing and processing of data in Office 365 documents. This decision underscores a growing recognition of the risks associated with large language models (LLMs) and their interaction with sensitive data. For business leaders and cybersecurity professionals, this serves as a critical wake-up call to the potential vulnerabilities in widely adopted AI tools. In an era where AI technologies are taking center stage, it is crucial to comprehend the implications of this ban in order to comply with it as well as to ensure that organizational data is secure.

Risks and Security Implications

The primary concern leading to the ban of Microsoft Co-Pilot revolves around the unauthorized access and misuse of sensitive information contained within Office 365 documents. LLMs like Co-Pilot process vast amounts of data, including potentially confidential or proprietary information, raising significant concerns about data privacy, intellectual property rights, and the security of personal and organizational information. The risks are manifold: from inadvertent data leaks to the potential for sophisticated cyber-attacks exploiting AI systems' data processing capabilities. Such vulnerabilities could lead to financial losses, reputational damage, and breaches of regulatory compliance, particularly in sectors governed by strict data protection laws.

Governance

In response to these challenges, governance frameworks must evolve to address the unique risks presented by LLMs and AI-driven tools. This includes establishing clear policies on AI usage, data sharing agreements, and rigorous oversight mechanisms. Organizations should conduct thorough risk assessments to understand their exposure and implement strategies that align with industry best practices and regulatory requirements. Effective governance also involves educating stakeholders about the potential risks and incorporating ethical considerations into AI deployment strategies.

Actionable Security Controls

As a business security and data protection professional, it is imperative to establish robust security controls to safeguard against the risks posed by external LLMs.

1. Pen Testing: Before AI tools can process sensitive information, implement thorough red teaming using the Mitre Attack Framework to identify risks and safeguard your data.
2. Access Controls: Strictly control access to sensitive data, ensuring that only authorized personnel and systems can retrieve or process such information.
3. Encryption: Encrypt sensitive data both at rest and in transit to prevent unauthorized access, even if data is intercepted.
4. Audit Trails: Maintain detailed audit trails of all interactions with AI systems, facilitating oversight and enabling quick responses to any irregularities.
5. Vendor Assessments: Conduct thorough security assessments of AI service providers to ensure they meet your organization's security standards.

Tools for Identifying, Detecting, and Protecting Confidential Data

Several tools can help manage and protect confidential data in the context of LLMs and AI processing:

• Data Loss Prevention (DLP) Software: DLP tools can monitor and control data transfer, preventing sensitive information from leaving the network.
• Cloud Access Security Brokers (CASBs): CASBs provide visibility and control over cloud services, including AI tools, ensuring compliance with security policies.
• AI Security Platforms: Specialized AI security platforms can analyze AI systems for vulnerabilities, monitor for malicious activities, and ensure data privacy compliance.
• Encryption Solutions: Tools that offer end-to-end encryption for data in transit and at rest can protect against unauthorized access, regardless of the data's location.

An example of one mitre attack technique to test. Thanks Jack Cardin

Tactics: Credential Access?and Privilege?Escalation

Technique: Input Capture (T1056) and Exploitation for Privilege Escalation (T1068): AI tools with access to user inputs might be exploited to capture sensitive information, including credentials and personal data. These vulnerabilities in AI platforms, coupled with newly found?user credentials, could be exploited to gain higher privileges, potentially allowing unauthorized access to sensitive data or systems.

The?best way to mitigate these tactics would be to not execute?any AI-generated code from an LLM that requires any sort of clearance to properly function. Additionally, do not provide any LLM personal passwords or credentials, as your responses could be monitored by bad actors through a backdoor. The safest course of action and easiest way to mitigate these attacks is to only request code from LLM's in small chunks for loosely coupled functions that require little to no security.

Jack Cardin came up with 9 other attack tests to review.

1. Technique: Phishing (T1566): Malicious actors could exploit AI-driven platforms to create highly sophisticated phishing campaigns, leveraging the AI’s understanding of language and context to trick users into divulging credentials or downloading malware.
2. Technique: User Execution (T1204): An attacker might leverage AI-generated content to trick users into executing malicious code, relying on the perceived legitimacy of AI-generated recommendations or documents.
3. Technique: Account Manipulation (T1098): Cyber adversaries could seek to manipulate or take over accounts connected to AI systems, ensuring persistent access to data processed by these tools.
4. Technique: Exploitation for Privilege Escalation (T1068): Vulnerabilities in AI platforms could be exploited to gain higher privileges, potentially allowing unauthorized access to sensitive data or systems.
5. Technique: Obfuscated Files or Information (T1027): AI systems could be used to create or modify malicious payloads in a way that evades detection by traditional security tools.
6. Technique: Input Capture (T1056): AI tools with access to user inputs might be exploited to capture sensitive information, including credentials and personal data.
7. Technique: Data from Information Repositories (T1213): If compromised, AI systems integrated with information repositories could be used to systematically collect and exfiltrate sensitive data.
8. Technique: Exfiltration Over Web Service (T1567): Data collected by AI tools could be exfiltrated through commonly used web services, blending in with legitimate traffic to avoid detection.