The TechnologIST | How can we mitigate malicious use risks of AI?

Welcome back to The TechnologIST! I'm Sophia Mauro . This month at IST:

• Read a?new report on AI risk reduction
• Meet the RTF Steering Committee , a group of senior-level leaders and experts in cybersecurity and technology law and policy, who provide support, guidance, and oversight on ongoing RTF lines of effort
• Check out Improving Nuclear Hotlines , a series of briefs?on crisis communications in country-specific contexts with guest authors from the Crisis Communications Resilience Working Group
• Watch AI Risk Busters , a virtual discussion?with experts from across the AI ecosystem

The Artificial Intelligence?Lifecycle

When you interact with an AI foundation model, whether to chat with a bot, generate imagery, or conduct forecasting, you are interacting with a system that has gone through a series of distinct stages, from development to deployment. Each of those stages poses nuanced risks—and presents opportunities for intervention.?

In December 2023, IST's AI Foundation Model Access Working Group, a group of AI developers, researchers, and practitioners helped to contribute to the report How Does Access Impact Risk , which categorized risks of AI and assessed how they change over a gradient of access. Following its publication, the group coalesced around the idea of focusing on risks and mitigation strategies that are both "upstream" and "downstream" across the AI lifecycle. As a result, IST developed an AI Lifecycle, featured in our latest report , which breaks down AI development into distinct stages—enabling an understanding of the risks that occur at each stage of the lifecycle, as well as the mitigation strategies that might be most effective.?

This month’s report takes a deep dive into the risk of malicious use, one of six categories of risk identified in the December 2023 report, and suggests how policymakers and regulators could apply the AI lifecycle framework to target the risk that AI is leveraged to harm individuals, groups, and society.

I sat down with report author and Artificial Intelligence Policy Associate Louie Kangeter and Senior Associate for Artificial Intelligence Security Policy Mariami Tkeshelashvili to learn more about the research that went into the report, an inside look at what's next for the AI Foundation Model Access Initiative, and key takeaways from our recent webinar launching the report.?

Q&A: Louie Kangeter and?Mariami Tkeshelashvili

The December 2023 report identified 6?categories of risk posed?by AI foundation models. This report zeroes in on the malicious use risk of AI, identifying strategies to mitigate the particular risks that are introduced when malicious actors use AI technologies to inflict harm. What prompted this specific area of focus?

Louie Kangeter : "We decided to begin applying our risk mitigation framework to malicious use for several reasons. Research and conversations with our working group indicated that malicious use was the risk most likely to materialize in real world applications first, and therefore needed to be a top priority. Indeed, we are already seeing malicious applications of AI technology, even with existing model capabilities, so it’s crucial to begin developing mitigation strategies for this category of risk. Additionally, malicious use is a broad category, covering many applications and use cases of AI, which makes it an excellent starting point from which to leverage the strengths of our framework."

Why create an AI Lifecycle?Framework?

Louie Kangeter : "We chose to develop the novel AI Lifecycle Framework because of the inherent complexity of the AI lifecycle. When providing actionable recommendations to policymakers, it is crucial to generate approaches that are both nuanced and broadly effective. In the case of AI, this means contextualizing risks and risk mitigations within a broader understanding of the function and structure of AI models. Our hope is that through utilizing the Lifecycle Framework, policymakers will be able to better understand the given stages of the AI lifecycle, and thus improve their capabilities to make more effective risk mitigations."

Some models are more open, providing access to model weights and architectures, whereas others remain highly restricted, limiting who can access the model and its components. How does the AI Lifecycle Framework intersect with this spectrum of openness?

Louie Kangeter : "The intersection between openness and risk mitigation is challenging. This is another reason why we are determined to utilize the Lifecycle Framework–no matter where a model sits on the spectrum of openness, key stages of the lifecycle apply. The degree of openness of a model interacts with the effectiveness of risk mitigations at given stages of the AI lifecycle. While certain risk mitigations may be less effective than others when interacting with openness, the lifecycle approach allows policymakers and developers to understand when and where it is crucial to implement risk mitigations. In essence, the lifecycle approach allows us to introduce the variable of ‘openness,’ and plan and act accordingly."

The report conducted a deep dive on five?specific types?of malicious use risks, exploring the overall trends, understanding how new AI technologies can exacerbate or affect these risks, and determining how, with extant and more advanced technologies, AI tools might be applied to each category. Of those, which was most concerning to you?

Louie Kangeter : "My concern is mainly focused on state actors. The powers bestowed by effective use of AI systems can and, in certain parts of the world, will be utilized by authoritarian regimes to consolidate power in a way which has never before been possible. Authoritarian regimes of the past required large bureaucracies and the use of military force to ensure their power. These same inefficient organizations often result in the downfall of the regimes that they were originally created to prop up. In a future where authoritarian regimes utilize AI systems to control their populations, it may become significantly more difficult to overthrow or dismantle power structures, leading to authoritarian regimes having significantly more capability to control their populations over long periods of time, leading to increased oppression and human suffering."

What’s next for the AI Foundation Model Access Initiative?

Mariami Tkeshelashvili : "We’re continuing to expand our work on the December 2023 report and plan to publish research focused on other risks like compliance failure, taking the human out of the loop, and reinforcing bias. Each of these risks poses unique challenges to national and human security. For instance, while various organizations and governments are working on introducing safeguards and regulations to ensure safe and secure AI, these measures are only effective when adhered to by both developers and users. However, enforcing some of the control mechanisms becomes increasingly challenging. We plan to explore why that is and what mitigation strategies can preempt compliance failure risk."

Mariami, you recently moderated a virtual conversation with stakeholders from across the AI ecosystem on the AI Lifecycle Framework and the potential risks of AI technologies. What was one of your key takeaways from the conversation?

Mariami Tkeshelashvili : "The discussion, among other things, highlighted how different threat actors misuse AI against Ukraine as Russia’s full-scale invasion continues and how Ukraine utilizes AI-powered tools to develop solutions to counter these risks. I believe that involving people who experience the consequences of the malicious use of AI in the conversation brings a unique perspective to any discussion about AI risks."

Elsewhere at IST

Introducing the Ransomware Task Force Steering Committee?

The Ransomware Task Force, launched in 2021 in response to the emerging national, economic, and international security threat posed by ransomware, continues work to tackle ransomware through working groups, convenings, and annual progress reports on the status of its 48 recommendations. Central to its success is the RTF Steering Committee , a group of senior-level leaders and experts in cybersecurity and technology law and policy,? who provide support, guidance, and oversight on ongoing RTF lines of effort.

Nuclear Risk Reduction in Context: A Country-Level Approach to Crisis Communications?

This series of publications written by members of the Crisis Communications Resilience Working Group assesses the state of crisis communications systems in nuclear-armed states, identifying political and technical barriers to cooperation in China, Russia, India, Pakistan, and the P3–France, the United States, and the UK. At what point during a crisis might states use a bilateral or multilateral crisis communications mechanism? Who would actually be responsible for adoption of that mechanism? Explore the series to learn more .

IST in the News

Jen Ellis calls out ransomware gang during hospital attack

The ransomware assault on London’s hospitals is a political protest, the group responsible claims. “Cyber-criminals like this gang lie routinely,” Ransomware Task Force Co-Chair Jen Ellis told the BBC. “Where they are from and why they have carried out the attack is secondary to the harm being caused right now to patients and hospital staff .”

Philip Reiner weighs on the U.S.-China hotline

A nuclear hotline connecting the United States and China might not survive a war, Christian Ruhl writes in the Bulletin of Atomic Scientists. According to IST CEO Philip Reiner, states are prioritizing the resilience of their internal communications systems instead. Communicating with an adversary in a crisis is “always an afterthought,” he explains .

What We're Reading

Want more tech and security content? Check out some of the ISTeam's favorite pieces from the past month:?

• As cyber threats evolve, artificial intelligence could “revolutionize” how we respond, Citi’s Janet Rathod and INSA’s Bishop Garrison write for Washington Technology . “AI has played a vital role in cybersecurity for decades, and the continued adoption of new AI technology can help organizations proactively detect and respond to threats with greater speed, accuracy, and efficiency,” they write.?
• The medical records of a “substantial proportion of people in America ” were stolen from Change Healthcare during a ransomware attack in February, the company announced recently. Change is in the process of notifying those affected, but cannot confirm what information was accessed.?
• A satellite crash could knock out a majority of technology we rely on every day. As low-earth orbit continues to fill with more and more debris, that risk increases, Marina Koen writes for the Atlantic .?
• A ransomware attack disrupted operations at hospitals in the UK early this month. National Health Service officials were warned about their systems’ vulnerabilities a year before the attack, the Independent reports .??
• Chinese President Xi Jinping recently called on the nation to mobilize “new quality productive forces.” Arthur Kroeber writes for Brooking Institution : “These slogans culminate a major shift in China’s overall economic policy direction that has been building for two decades, and that has accelerated noticeably in the last five or so years of Xi’s administration.”
• Anthropic shared insights from red teaming approaches they’ve used to test their AI systems and offered their policy recommendations, which include funding NIST and other independent government bodies and non-profit organizations “that can partner with developers to red team systems for potential risks in a variety of domains .”

The Institute for Security and Technology designs and advances solutions to the world’s toughest emerging security threats. It is a nonpartisan, nonprofit organization?based in the San Francisco Bay Area dedicated to solving critical international security challenges through better technology and policy. Donate today to support our mission.

To receive The TechnologIST in your inbox, subscribe to our mailing list .