AI Safety: The Missing Piece in the AI Development Puzzle

Bridging the Divide: Translating AI Safety Research into Actionable Insights

If you are like me, you stumbled here because you have seen cases where Generative AI products have failed in the market, like the Air Canada chatbot. Or maybe you are an Engineer or Product Manager wondering if the LLM application you are deploying is safe. Either way, you are at the right place since this blog will go over what AI Safety will look like and what you can do to have safe models in production.

Understanding AI Safety for Engineers and Product Managers

As AI systems, particularly LLMs like GPT, become integral to products, their ability to act autonomously and influence real-world outcomes underscores the need for rigorous safety protocols. AI Safety extends beyond ensuring that an AI system functions correctly and efficiently; it encompasses the ethical implications, potential for misuse, and unintended consequences of AI technologies.

AI safety also involves evaluating the societal impact of AI systems, ensuring that they do not perpetuate or exacerbate unfairness, discrimination, or other social harms. For product teams, this means incorporating a multidisciplinary approach that includes ethics, policy, and user safety from the inception of product development. It's about foreseeing potential misuses and designing safeguards to prevent them. This proactive stance on AI safety can help in building trust with users and ensuring long-term success in the marketplace.

Moreover, with the increasing complexity of AI systems, engineers and product managers must be equipped with the latest knowledge and tools to evaluate AI risks accurately. This includes understanding the nuances of AI behavior, such as data biases, model transparency, and explainability, which are critical for diagnosing and mitigating safety issues.

AI Safety: An Afterthought in the Rush to Market

In the race to leverage the capabilities of LLMs for commercial products, AI safety often becomes an afterthought. The eagerness to deploy AI-powered solutions and capitalize on their potential economic benefits leads many teams to prioritize speed over safety. This rush can overlook the importance of thorough testing, leading to unforeseen issues like model hallucinations, where AI generates incorrect or nonsensical information.

This oversight is not just a technical flaw; it represents a significant risk to user trust and product reliability. The consequences can range from minor inaccuracies to severe misinformation, potentially causing real-world harm or influencing decisions based on false premises. This challenge is exacerbated by the opaque nature of AI models, which can make diagnosing and correcting such issues more difficult.

Addressing AI safety from the outset requires a shift in mindset, recognizing that the long-term success of AI products depends not only on their innovative capabilities but also on their safety and reliability. It calls for embedding AI safety principles into the product development cycle, from design to deployment, ensuring comprehensive testing and readiness to address any issues proactively.

A snapshot into where AI Product Development is Today

The current product development landscape, especially with the integration of LLMs, often follows a "build fast, fix later" approach. Here's a typical product team's roadmap:

• Phase 1: Enthusiasm and Aspiration - The team discovers the power of AI and dreams up a product that's sure to be the next big thing. Deadlines are ambitious, and the potential ROI looks fantastic on paper.
• Phase 2: Fast-Track Development - With an eye on the prize, development shifts into high gear. Testing is cursory—"It works on my machine!" becomes the team's unofficial motto. Safety checks? They'll fit those in later, there's no time to lose!
• Phase 3: Deployment Delight - The product hits the market with a bang. Initial reviews are glowing. The team celebrates—success is imminent!
• Phase 4: The Hallucination Hangover - Users start reporting oddities. The AI isn't just thinking outside the box; it's in another dimension. The product, it turns out, wasn't quite ready for prime time.
• Phase 5: Retrofitting Reality - Now comes the scramble to address the unforeseen. Sleepless nights are spent patching, testing, and retesting. The focus shifts to what should have been done in the first place: ensuring AI safety and reliability.

This roadmap, while exaggerated, highlights the pitfalls of sidelining AI safety in the development process. It serves as a cautionary tale for teams eager to leverage AI without due diligence on safety considerations.

The State of AI Safety Research

The AI safety research landscape is diverse and expanding, with various organizations addressing different facets of AI safety. The "Map of AI Existential Safety" document showcases a broad spectrum of entities focused on research, policy, and practical solutions to AI safety challenges.

Organizations like the Future of Life Institute, OpenAI, and DeepMind are at the forefront of technical safety research, developing frameworks and techniques to ensure AI systems operate within intended boundaries.

Nonprofits and think tanks, such as the Center for Security and Emerging Technology (CSET) and the AI Policy Institute, contribute to policy and governance discussions, emphasizing ethical considerations and societal impacts of AI. Meanwhile, entities like AI Safety Camp and the Machine Intelligence Research Institute (MIRI) support emerging researchers and foster collaborations to tackle unresolved safety problems.

Despite the growth in AI safety initiatives, gaps remain. Many efforts are siloed, leading to duplication of efforts and a fragmented understanding of the AI safety landscape. Bridging these gaps requires greater collaboration and sharing findings and methodologies across institutions. Furthermore, there's a pressing need for more applied research that translates theoretical safety concepts into practical tools and strategies that AI developers can implement.

For example, recent studies like "How predictable is language model benchmark performance?" by Epoch and "The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning" from the Centre for AI Safety Research have advanced our understanding of how AI models perform across different tasks and introduced methods to mitigate risks associated with AI misuse.

Despite these advancements, challenges remain in translating this wealth of research into practical, industry-ready solutions. Often, the findings from safety research, while groundbreaking, do not directly translate into tools or practices that developers and managers can readily implement. For instance, while the work by Epoch offers insightful predictions on AI capabilities, it lacks direct applicability in everyday AI system development. Similarly, although the WMDP Benchmark introduces innovative unlearning methods, the broader adoption and implementation of such techniques across the industry are yet to be seen.

The Solution to Bridge the Gap

The journey towards integrating AI Safety research into practical applications within the industry requires a dual approach.

Firstly, we need to recognize companies such as Lakera, Mindgard, and Athina that are pioneering this movement by focusing on developing tools and methodologies that directly address the safety concerns in Large Language Models.

For example, Lakera Guard analyzes LLM systems and recognizes prompt injection attacks, PI information, or unknown links. Furthermore, they have a Lakera Red Team, which works with companies to find vulnerabilities in their Generative AI Products. Their work is just the start of safety being taken seriously by the industry. In the future, if research can be the cornerstone of AI Safety, more products with AI Safety elements can come to the market.

In the coming weeks, we will look into more tools and products like Lakera, creating a space for AI safety and security in the industry

The second approach is to amplify the conversation in more accessible and diverse forums, highlighting the human element behind the technology.

Beyond discussing models and algorithms, sharing personal stories and experiences from those working at the forefront of AI Safety provides invaluable insights.

This approach humanizes the technical discourse, making it more relatable and understandable to a broader audience, including policymakers, developers, and the general public. By elevating these stories, we underline the collective responsibility of ensuring AI technologies are developed with ethical considerations and safety at their core. Such narratives inspire action, foster a safety culture in AI development, and encourage the industry to adopt best practices prioritizing human well-being.

Recognizing the gap in connecting AI Safety research with its human impact, "Humans of AI Safety" is an initiative to spotlight individuals and their contributions to making AI systems safer.

This week, we're featuring Akash Kunde from Apart Research Labs, highlighting his significant work in AI Alignment Research for Multi-Lingual Models. Kunde's efforts exemplify the critical need for AI technologies to align with human values across diverse linguistic and cultural contexts, ensuring that AI benefits are universally accessible.

Spotlighting his work underscores the importance of product developers and the wider industry considering ethical implications and safety in AI deployment. Focusing on the people behind the research, "Humans in AI Safety" aims to inspire more inclusive and thoughtful approaches in AI development.

Working in AI Safety and want to talk about your journey?; Reach Out!

In the coming weeks, "Humans of AI Safety" plans to explore more avenues for effectively translating AI Safety research into industry practices.

This endeavor will involve identifying emerging themes and innovative solutions across different domains of AI Safety, such as privacy preservation, fairness in algorithms, and strategies to mitigate AI systems' unintended consequences.

By showcasing the work of researchers and practitioners making strides in these areas, the initiative aims to foster a collaborative environment where ideas can be exchanged freely, and innovations can be adopted swiftly.

Reach out to us