Navigating the Risks of AI: A Path to Public Trust

The rapid advancement of artificial intelligence (AI) systems like self-driving cars and large language models has captured the public's imagination. As AI expands into sensitive domains like defense, healthcare, and finance, maintaining trustworthiness becomes crucial. AI risks arise from faulty logic and malicious attacks. In addition, all types of AI systems can present unique risks due to the extensive and complex supply chain required and the statistical nature of training for both deterministic and stochastic models. Earning public confidence requires documenting and disclosing an organization’s efforts across the AI lifecycle – from curating training data to monitoring systems post-deployment. Trust emerges when organizations embrace their responsibility to operate AI safely, legally, ethically, and equitably and communicate their efforts to the AI systems’ target audience.

Deterministic vs. Stochastic Systems

Traditional computer software is deterministic, meaning it always produces the same output given the same input based on predefined logic. In contrast, AI systems can exhibit either deterministic or stochastic behaviors. Deterministic AI reliably generates identical outputs for a given input, while stochastic AI incorporates randomness and may produce varied outputs for the same input. For example, the same prompt to two different large language models will produce two different responses.

The public's use of stochastic AI could result in diminished trust of AI in certain contexts. This is because stochastic systems introduce additional uncertainties from the randomness in training, outputs, and operations. However, it's important to note that both deterministic AI and stochastic AI still present significant risks around data, bias, security, and more. Ultimately, all modern AI systems - both stochastic and deterministic - entail complexities surrounding transparency, interpretability, and safety.

Managing the Risks of Artificial Intelligence?

Pursuant to the National Artificial Intelligence Initiative Act of 2020, goal to promote trustworthy and responsible AI, the National Institute of Standards and Technology (NIST), a US agency experienced in providing guidance on standards best practices for new technologies, published the AI Risk Management Framework (NIST AI 100-1) (NIST RMF) which provides a practical approach for organizations to govern AI risks, map risks in context, and measure them.? It also has an accompanying playbook.

The guidance ?is voluntary and seeks responsible AI. It covers the life cycle of the AI system with two basic parts: the Framework and the Core.?

The Framework

The Framework outlines seven key characteristics of trustworthy AI:

1. Valid and reliable,

2. Safe,

3. Secure and resilient,

4. Explainable and interpretable,

5. Privacy-enhanced,

6. Fair with harmful bias managed, and

7. Accountable and transparent.

?

Accountability and transparency span across the other criteria. Crucially, the Framework emphasizes engaging diverse perspectives across the AI lifecycle, from developers to users to affected communities, to ensure AI systems are developed and used in a socially responsible manner. It also integrates regular test, evaluation, verification, and validation (TEVV) to assess evolving risks. Adopting this lifecycle approach will enable organizations to tap AI's benefits while navigating its downsides. (See Appendix A for the actors involved in the AI life cycle.)

The Core

The core consists of four overarching functions to manage risk control activities:

·?????? Govern – Focuses on ?organizational policies, accountability structures, and risk culture.

·?????? Map - Identifies risks, impacts, and AI system requirements.

·?????? Measure – Analyzes and evaluates risks quantitative and qualitative methods.

·?????? Manage - Prioritizes risks to develop mitigation plans and responses.

?Safeguarding AI Systems from Adversarial Threats

Artificial intelligence promises to transform industries, but as said, poses novel risks. In particular, adversarial machine learning attacks could undermine public trust in AI. NIST has published a voluntary guidance categorizing different adversarial attacks and recommending defensive techniques, Adversarial Machine Learning A Taxonomy and Terminology of Attacks and Mitigations NIST AI 100-2e2023 ipd, (NIST AML). Understanding these emerging risks is key for organizations seeking to responsibly adopt AI.

Vulnerabilities and Attacks

Adversarial machine learning involves malicious activities aimed at corrupting AI model integrity or behavior. NIST identifies several types of threats:

·????? Evasion Attacks: Adding small manipulations to model inputs causing misclassification. For example, altering a few pixels can lead an image classifier to mistake a turtle for a pedestrian.

·????? Data Poisoning: Injecting tainted data into model training degrades performance. Poisoning medical images during training could degrade disease diagnosis.

·????? Model Theft: Extraction attacks copy proprietary model logic and parameters. Stolen models present intellectual property theft risks.

·????? Model Inversion: a technique that can be used to infer private data from model outputs.

·????? Logic Corruption: Manipulation of models to force unintended and potentially harmful behaviors.

These threats highlight AI's unique vulnerabilities. Whereas bugs arise from faulty code, adversarial risks exploit the inherent statistical nature of machine learning.

Protection and Defenses

NIST AML recommends varied defenses to secure AI, including:

·????? Adversarial Training: Augment model training with adversarial examples to increase robustness against evasion.

·????? Outlier Detection: Monitor systems at runtime to detect unusual inputs and behaviors indicating potential attacks.

·????? Sandboxing: Isolate systems and restrict access to sensitive assets to minimize attack surface.

·????? Data Minimization: Employ encryption and masking techniques that preserve privacy and prevent inversion.

Organizations should also foster coordination between security and AI teams, conduct penetration testing, and implement cybersecurity best practices tailored to AI's novel risks.

Adversarial threats are an inevitable consequence of AI's growing pervasiveness. NIST's guidance empower developers, users, and regulators to collaborate preemptively in securing AI and upholding public trust.

Organization accountability for trustworthiness in an organization

Security operations centers (SOCs) are traditionally responsible for information security of an organization's standard IT systems. Given AI's emerging risks, SOCs are a logical group to focus efforts to achieve AI trustworthiness. However, this would require enhancing SOC capabilities to manage novel AI risks.

The NIST AI RMF introduces core functions like Govern, Map, Measure, and Manage specifically focused on AI systems, beyond SOCs' current cybersecurity duties. To support these new functions, SOCs may need additional staff with specialized AI/ML expertise, rather than just general security skills. Personnel should have knowledge to uphold the NIST criteria for trustworthy AI and perform activities like bias testing, adversarial robustness evaluation, and AI model risk assessments.

With appropriate skilled resourcing and expansion of responsibilities to include AI specific risks, SOCs can leverage their organizational security experience to ensure accountability for trustworthy AI. But they cannot treat AI systems the same as traditional IT - the technology and threat landscape calls for dedicated focus on managing AI's novel vulnerabilities.

Documentation

SOC teams currently produce periodic reports summarizing the organization’s security posture and incidents, typically for senior management. These existing processes provide a foundation for AI trustworthiness reporting, but some adaptations for AI would be beneficial. AI transparency disclosures may require more understandable, non-technical presentations for broader audiences like investors, regulators, and consumers (users). The focus should expand beyond technical and financially material impacts to also cover social, political, legal and ethical risks from AI systems. While SOCs report periodically, AI incidents may necessitate prompt reporting given public safety implications. Disclosures should not just flow to senior management but also to stakeholders like management committees, boards of directors, customers and, when appropriate, user groups. Leveraging SOCs’ expertise in documentation is prudent, but reporting specifically on AI trustworthiness necessitates broadening accessibility, audience, timeliness, and scope compared to traditional infosec reporting. With some modifications, SOC documentation capabilities can be extended to cover AI risks.

SEC Disclosure Rules

The SEC's rule 17 CFR 229.106 requires public companies to disclose cybersecurity risks, governance, and incidents. However, this rule covers only some of the criteria in the NIST AI Risk Management Framework for trustworthy AI. It does not require disclosure of most of the risks incident to AI systems.

The SEC rule focuses primarily on cybersecurity threats to system integrity, not comprehensive risks to AI safety and society. It mandates disclosing:

·????? Processes for assessing cybersecurity risks (106(a))

·????? Material impacts from cybersecurity threats (106(b))

·????? Board oversight of cybersecurity risks (106(c)(1))

·????? Management's role in assessing cyber risks (106(c)(2))

In contrast, as previously stated, the NIST Framework has seven pillars: valid/reliable, safe, secure, explainable, privacy-enhanced, fair, and accountable AI. The SEC rule explicitly relates to the "secure" aspect only.

Importantly, the rule defines cybersecurity threats and incidents narrowly as events that could adversely impact confidentiality, integrity or availability of information systems. However, many risks to trustworthy AI are not "events" - like flawed data or algorithms.

While the SEC rule promotes transparency on cyber risks, it does not cover the full scope of risks that should be disclosed for responsible AI. Broader evaluation of risks to society is required.

In summary, while the SEC rule supports cybersecurity, it lacks a comprehensive perspective on AI risks across areas like fairness, interpretability, and safety. Organizations need to go beyond compliance to fully assess and disclose AI risks responsibly.

Conclusion

As artificial intelligence expands into sensitive domains, maintaining trustworthiness is imperative. AI introduces novel vulnerabilities compared to traditional software, arising from the statistical nature of machine learning and the extensive supply chain required. Deterministic and stochastic AI both warrant proactive risk management, though stochastic systems may face additional uncertainties.

Organizations have a responsibility to operate AI safely, ethically, and equitably. Embracing accountability and transparency will build public confidence. Frameworks like the NIST AI RMF provide helpful guidance on managing risks across the AI lifecycle. While regulations around cybersecurity disclosures promote some transparency, even broader communication of AI risks is needed.

Trust emerges when all stakeholders work collectively to realize AI's benefits while navigating its threats. Developers, companies, regulators, and the public each play a role in fostering responsible AI adoption. With collaborative, human-centric risk management centered on safety and ethics, AI can transform society for the better. But earning public trust requires proactive mitigation of AI's novel risks.

#ResponsibleAI #EthicalAI #TrustworthyAI #AISafety #SafeAI #AIRisks #AIGovernance #NISTAI #AIDisclosures

#AISecuritiesLawDisclosures

APPENDIX A

Actors in AI Lifecycle

Chart in NIST AI RMF

?

paired with comprehensive transparency.