The TechnologIST: Key takeaways and themes from IST's work on artificial intelligence

Welcome back to The TechnologIST, IST's monthly newsletter featuring exclusive interviews with our experts, latest news, and recommended reading.?I'm Sophia Mauro , Deputy Director of Strategic Communications at IST. This month, we'll hear?from Alexa Wehsener, a key contributor to our work on AI over the last 4+ years.?

The following is a preview of our latest newsletter. To subscribe and make sure you don't miss the full story, click the link below:?

"The" AI Spring?

“Years from now, we will likely view Spring of 2023 as “the” AI Spring, distinct from the oft-discussed AI Springs of the past,” write Megan S. , Zo? Brammer , and Alexa Wehsener in our latest blog,?Catalyzing Security in AI Governance. “Whereas the use of social media and enhanced Internet connectivity during the Arab Spring helped enable political movements years in the making, this current AI Spring evidences significant accelerations in technological advances that have also been underway for decades.”

Like many governments, individuals, and businesses across the world, we at IST are reflecting on the implications of this current, markedly more impactful?AI Spring. We come to this work with a strong foundation in research and convenings around artificial intelligence and machine learning; from recent efforts sponsored by the U.S. Department of State to investigate the?integration of artificial intelligence into nuclear domains?and its impact on strategic stability to early workshops hosted by the Lawrence Livermore National Laboratory on?AI and human decision-making, we are no strangers to this field. We are currently moving forward with additional collaborative efforts to catalyze security in AI governance and champion digital security, acknowledging the need to seize the incredible opportunities presented by these new capabilities, but also to be patient and earnest in identifying and devising solutions for the very real risks we must collectively face.

Alexa on AI

To unpack IST's role in this field, I spoke with?Alexa Wehsener, Deputy Director of Defense Strategy and Research and a major contributor to our work on AI over the last 4+ years. She has led full life cycle research efforts, is a co-author of multiple reports on AI and has convened multiple workshops with key policymakers and technical researchers in the field.?

Q: What exactly is AI? Why is defining what it is–and isn’t–important?

"AI is the umbrella under which various types of methods (neural networks, language processing, knowledge representation, etc.) and learning (machine learning, reinforcement learning, deep learning, etc.) sit. Arguably more important than defining what AI is and isn’t is defining what specifically is being referenced and for what application. When discussing AI and thinking about how it could be integrated and deployed, we should focus on particular models or applications. The conversation is really most useful when we’re specific about both. My bottom line is: We should think about AI techniques as tools. In what way can these tools, when applied, be harmful? To whom? In what context do they create new security challenges or exacerbate existing ones? When these tools are trained are they done so in a way that aligns with human values? The AI safety and alignment community engages in amazing work in this vein."?

How does this question of understanding how AI is being referenced and for what application come into play in a national security context??

"In a national security context all of these questions I posed above are important, among others. Over the past 10 years, we have witnessed a pivot in our everyday lives–whether we know it or not–due to evolving advancements in AI and ML, including in R&D and applications of techniques to civilian and government sectors alike. This means that our national security infrastructure must also be able to pivot to recognize that AI runs through every aspect of our society, enabling the emergence of novel attack vectors and unforeseen consequences. Baseline understanding of AI methods and their intended applications is a crucial first step in mitigating the risks while capitalizing on the opportunities that AI-enabled systems can provide the national security community."

You mentioned that there’s been this ‘pivot’ in AI developments over the last 10 years, including expanding research and development, as well as new contexts in which AI is being applied. What has changed??

"Today, there is considerable focus on the advancements of large language models, training environments, and the use of open source for tooling. For me, what is really interesting today vs. the past 10 years is that the public is now able to?intentionally?interact with many of these advancements. Many of the tools and programs we interact with on a daily basis have long incorporated AI into their platforms. Although the general public previously engaged with AI almost daily, they had little idea. The public release of large language models like ChatGPT and Baud (among others) and the resulting media coverage mean that now the everyday citizen is aware of the mechanism driving the output: when we type a question into ChatGPT models, we understand and accept that we directly engaged with an AI model and that it is actively learning from our inputs.?

This public awareness and intentionality behind interaction with AI is driving new questions about frameworks and governance surrounding AI usage. What does the integration of AI across so many sectors, both private and public, mean, especially when advancements in AI research at the edge are occurring at voracious speeds and are released to the public or are open-source? What are the tools and what are the architectures that need to be established for democratic governments to keep pace? And finally, what are the opportunities and risks inherent to not only the models themselves but to their application, especially when we lack current guiding principles of how to ensure equitable and safe integration of such learning techniques and their development?"

If you were to pull a theme out of IST’s work on artificial intelligence thus far, what would it be? What have been some of your key takeaways from your research??

"A common thread throughout all of our research is asking?where?and?how?novel AI approaches and techniques may play a role in their given application environment. At IST, we have always sought specification as to the exact learning techniques being discussed and their application. Our approach encourages dialogue around tangible possibilities, rather than?speaking in big picture, high-level ways. To do so, we not only bring in leaders from industry and the U.S. government to our closed door discussions, but also researchers who work in the technical weeds and at the cutting edge.?

Key takeaways:

• Technical insight in governance: Those at the technical edge must continue to play a role in the creation of governance frameworks. It is vital that spaces exist in which candid exchange between researchers and the U.S. government can continue. Moreover, those facilitating the conversations must do so from a place of neutrality.
• Proactive policy approaches: Proactive consideration of policy tools is crucial as novel technologies emerge and existing technologies continue to evolve.?
• Cross-domain applications: Insights on opportunities, vulnerabilities, and challenges from one application area such as nuclear command, control and communications systems can often be applied to other safety critical applications. For those thinking about AI applications in the military domain, there are many industry lessons learned in safety-critical areas such as aviation and autonomous vehicles on human-machine teaming, run time monitoring, calibrating verifiable trust, etc. We should be sure to continue to intentionally seek out and learn from cross-domain application of similar techniques applied to similar safety-critical tasks and integrate them into testing, evaluation, verification and validation (TEVV) practices.?
• Confidence building measures: Creating technically informed confidence building measures (CBMs) that are available on sliding scales provides policymakers with technology governance inspiration and de-escalation options. Doing so as the technology is emerging or has yet to emerge enables decision makers to lead in the creation of frameworks and norms for use in international fora. Process matters in diplomacy and it can genuinely shape an outcome—ensuring that diplomats are armed with technically informed upper and lower bounds and that they understand the technical and geopolitical ramifications will encourage early inclusion of allies and potentially even adversaries in international dialogues on AI advancements.?
• The power of convening: Mapping and convening of communities of interest related to advanced AI techniques, methods and applications as well as AI R&D enables cross-pollination of technical knowledge between technical communities of interest and the U.S. government."

“We should think about AI techniques as tools. In what way can these tools, when applied, be harmful? To whom? In what context do they create new security challenges or exacerbate existing ones?”

-?Alexa Wehsener,?Deputy Director of Defense Strategy and Research

Updates from IST

“De-risking” beyond China: Prescriptive Diversification and New Pathways for Allied Engagement

The latest G7 Hiroshima Leaders' Communique notably included a shift from "decoupling" to "de-risking" with China.?According to IST's Katherine Schmidt , this new rhetoric suggests "implicit alignment" between U.S. China policy and cooperation with foreign allies, as well as positive suggestions for industry to diversify investment and supply chains.

Impact, yearly financials, and a by-the-numbers look at 2022 in IST's Annual Report

IST released its 2022 Annual Report,?a look at the last year of impact and success across our organization.?In 2022, we published 21 reports, briefings, and one pagers, engaged with over 3,440 people at our public-facing events, and were cited in over 160 articles. We testified before Congress, participated in 45 events, and held closed-door briefings with entities worldwide.

IST in the News

Zo? Brammer unpacks ransomware payment cycle, keys to adding friction, and encryption process

On?the Global Initiative against Transnational Organized Crime 's podcast, Deep Dive,? Zo? Brammer discussed her work on mapping threat actor behavior in the ransomware payment ecosystem. "The ultimate goal is to add friction to this ecosystem, and by friction I mean disruption. How can we make it more difficult for actors to carry out attacks and less profitable for them to do so?"

Megan Stifel on concerning trends in the ransomware ecosystem

Speaking with Decipher's Lindsey O'Donnell-Welch , Megan S. said shifting tactics, techniques and procedures, dumping of confidential information, and integrity attacks that manipulate the data to make it more problematic are some of the areas she's watching. "Thinking about where they will evolve next and trying to keep up or stay a step ahead are some of the things that keep me up at night."?

What We're Reading

Want more tech and security content? Check out some of the ISTeam's favorite pieces from the past month:?

• During Secretary of State Blinken’s visit to Beijing this month, China reportedly rejected an offer to set up a direct?military line of communication?between the two.?But relations between the two are not fully frozen—there remain some open lines of diplomatic communication.?
• As companies and governments look for frameworks to govern artificial intelligence, an article in the Bulletin of the Atomic Scientists argues?nuclear governance is not a good model?to use: “It took decades to build an effective system of control for atomic energy even with a common view of the risk.”?
• As the world faces a transition towards clean energy and away from oil, China is set to play a major role: they currently manufacture more than 80% of global solar panels.?The West and China "will have to find ways to co-operate alongside the imperative to compete,"?argues Graham Allison.??
• Large language models may allow?people without formal laboratory training “to identify, acquire, and release viruses?highlighted as pandemic threats in the scientific literature,” suggest researchers. They urge caution with dataset creation, evaluations of the models pre-release, and universal DNA screening.?
• Is there a path forward for the use of artificial intelligence in autonomous cyber defense??CSET researchers identified reinforcement learning as one promising approach.
• Venture capital firms in Silicon Valley?sent an open letter to Secretary of Defense Austin?calling for an overhaul in the military procurement process.?
• A?security researcher made the case for a switch?from the Common Vulnerability Scoring System as a metric for assessing risk of security vulnerabilities to the Stakeholder-Specific Vulnerability Categorization. Given its codification based on likelihood, impact, severity, and risk, the SSVC model “will help organizations both effectively prioritize mitigation efforts and serve as a basis to measurably reduce business risk.”?

The?Institute for Security and Technology?designs and advances solutions to the world’s toughest emerging security threats. It is a nonpartisan, nonprofit network based in the San Francisco Bay Area dedicated to solving critical international security challenges through better technology and policy.?Donate today to support our mission.

Thanks for reading The TechnologIST!?To be sure you don't miss the next edition,?subscribe here.