AI Safety Institute

The UK-US agreement on AI safety is a significant moment for the AI Safety Institute and for the development of global safety standards on AI.? Here’s what it involves: The US and UK AI Safety Institutes will jointly test advanced AI models. We will also share research insights, share model access, and enable expert secondments between the Institutes. This will allow us to develop a shared framework for testing advanced AI and international best practices for other countries to follow. By working together, the UK and US can minimise the risks of AI and harness its potential to help everyone live happier, healthier and more productive lives. Find out more: https://lnkd.in/eX4bsy6G

We've released a technical report detailing our pre-deployment testing of Anthropic's upgraded Claude 3.5 Model with the U.S. AI Safety Institute. Read our blog for a high-level overview. https://lnkd.in/e3ZaES9x

Our new paper on safety cases, in collaboration with Centre for the Governance of AI (GovAI) shows how it’s possible to write safety cases for current systems, using existing techniques. We hope to see organisations using templates like this for their models. https://lnkd.in/ew2D8ZmK

AI is moving fast - and so are we. A year since our founding at the AI Safety Summit, we've built a world leading organisation dedicated to understanding AI threats. Today, we share more on what we’ve built, how we're working to ensure that AI’s harms don’t prevent its potential, and where we’re headed. And there’s more! To mark our anniversary, we're releasing InspectEvals – a new repo of dozens of high quality open-source evaluations for safety research. ? We’ve partnered with Vector Institute and Arcadia Impact to develop evals on coding, math, cybersecurity, safeguards and more. InspectEvals includes leading benchmarks and several agent benchmarks, which can now be run against any model with a single command. Read our blog post to get started! ??? https://lnkd.in/eYKxe8tD

We’re looking for talented individuals and organisations to help us build evaluations. We’ll reward bounties for new evaluations and agent scaffolding tools that assess the risks of autonomous AI systems. Find out more and apply by 30 November: https://lnkd.in/eykusYKb

We're bridging the gap between theory and practice in the rapidly evolving field of AI safety. If you're an academic looking to collaborate on cutting-edge AI safety research, we want to hear from you! ? We're now accepting Expressions of Interest from researchers and academics interested in collaborating on key research areas like safeguards, science of evaluations, and safety cases. Find out more:? https://lnkd.in/eyPXbnKT

Our Systemic AI Safety Fast Grants scheme is open for applications. Working closely with UK Research and Innovation and The Alan Turing Institute, we’re supporting projects that advance this new area of research. Moving beyond the safety of models themselves, systemic safety focuses on the broader resilience of society and critical infrastructure to AI-related hazards. We expect to offer around 20 grants up to ￡200,000 each in this phase, and particularly welcome projects bringing together researchers, industry and civil society. Find out more information and full guidance on how to apply: https://lnkd.in/euCYmY8S

Many new LLMs now possess agentic capabilities - they can use external tools and complete tasks for users, instead of just interacting as chatbots—this creates a new set of risks which require new evaluations and mitigations. ? We've published new research in collaboration with Gray Swan AI measuring the harmfulness of LLM agents. ? The paper highlights the need for LLM safety evaluations that focus on unique harms from AI agents with access to external tools. ? AgentHarm, a novel dataset, is designed to measure unique agent harms. It's easy to run, comprehensive & reliable. ? Read the full paper: https://lnkd.in/eHYM3P_k View the dataset: https://lnkd.in/ey-geJM4

Should AI systems behave like people? AI systems that can interact with us naturally are getting better. Humanlike AI could result in more engaging systems, but also poses safety risks and raises ethical questions.? Our new study asks the UK public what they think about humanlike AI. We asked the UK public for their thoughts on a range of topics, including:? ? Should a person know when they’re talking to an AI system??? ? Should chatbots express emotions like joy, sadness or loneliness??? ? Is there a world where humans and AI systems form meaningful relationships?? Here’s a snapshot of our findings:? ? Most people think AI should reveal itself not to be human, but didn’t mind realistic conversation ? Most did not want AI to express emotions, unless they were idioms (like “I’m happy to help”) ? Most people did not think that people could or should form personal relationships with AI systems Understanding the public’s view on humanlike AI helps to ensure that what counts as “safe” AI behaviour isn’t decided by researchers or policymakers alone.? This is key as we work with the wider AI community to minimise potential harm to the public from AI. https://lnkd.in/eWKUZjVw