Constitutional AI: Embracing Bias in the Quest for AI Alignment

In the evolving landscape of artificial intelligence governance, Anthropic's recent policy memo on Constitutional AI (CAI), Constitutional AI: Harmlessness from AI Feedback, caught my attention. As someone peripherally involved in early responsible AI policy research, I've seen my fair share of approaches to AI alignment. But this one? It's different, and it's got me thinking.

The Current State of AI Alignment

Many current approaches to AI alignment remind me of filling out forms and pledging allegiance to a particular agenda. It's bureaucratic, it's top-down, and frankly, it's ripe for breeding resentment and backlash. As a self-proclaimed Protopian big-L Liberal, it pains me to see my fellow bleeding hearts fall into this trap.

Enter Constitutional AI

Anthropic's Constitutional AI proposes giving AI systems a set of principles - a "constitution" - to evaluate their own outputs. It's an elegant idea, reminiscent of Enlightenment thinking. But as someone triggered by the word "constitution" in today's climate, I can't help but wonder: Are we setting ourselves up for the same pitfalls we've seen with actual constitutions?

The Tiki Toss Paradox

To illustrate the complexities of AI alignment, consider a popular bar game called "Tiki Toss." Players swing a ring on a string, aiming to catch it on a hook. At first glance, this seems like a perfect metaphor for AI alignment - we want our systems to consistently hit the target, right?

But here's the rub: in the real world of AI development and governance, "winning" this game every time would mean we all lose. An AI system so perfectly aligned with a particular set of goals or biases that it never fails to "catch the hook" would be inflexible, lack nuance, and potentially amplify harmful biases.

The Tiki Toss metaphor, while imperfect, illuminates a crucial point about AI alignment. Our Founding Fathers did their best with the tools at hand, even innovating with the amendment process. But their "Natural Intelligence" constitution, brilliant as it was, is like stone knives and bearskins compared to the potential of a well-crafted AI Constitution.

An AI Constitution has the potential to be far more robust, adaptable, and comprehensive than our current framework. It could process vast amounts of data, consider countless scenarios, and update in real-time to address new challenges. However, we're not quite there yet. The current state of the art in AI, while impressive, isn't ready to take on the full complexity of constitutional governance.

But here's the kicker: like self-driving taxis and humanoid robots, Constitutional AI will likely be ready sooner than most people expect. The pace of AI development is breathtaking, and the tools we're creating today may soon seem as quaint as quill pens to future generations.

As we stand on this precipice of a new era in governance and AI alignment, our task is clear. We must learn from the wisdom of our Founding Fathers while leveraging the incredible potential of AI. The goal isn't to create a perfect, unchanging system, but to develop an AI Constitution that can grow, adapt, and protect our values in ways we can barely imagine today.

The Promise and the Necessity of Bias

On reflection, perhaps the key isn't to avoid bias, but to embrace it thoughtfully. After all, when did "bias" become such a dirty word? Bias is just another term for "baseline" - it's the midpoint of our Overton Window, the DC current in a tape head (to use an analog reference in our digital world).

The US Constitution had a "bias" toward liberty and human rights. Yes, their definition of "human" was woefully incomplete, invented before Darwin and before Jefferson wrote his Bible. But the bias itself wasn't the problem - it was the foundation for progress.

In the same vein, Constitutional AI isn't about eliminating bias, but about consciously choosing and implementing it. We want our AI systems to have a bias towards beneficial outcomes for humanity. The challenge lies in defining that bias carefully and comprehensively.

The Perils of Misalignment: A Personal Anecdote

As I was crafting this post, I encountered a perfect example of how biases can go wrong in AI systems. I attempted to generate an image using ChatGPT to illustrate the concept of Constitutional AI. Despite providing explicit instructions, the AI not only rewrote my prompt but also blatantly ignored my strong directive to avoid text in the generated image. Here's the prompt that GPT-4 rewrote for me:

Create an Early American political cartoon style landscape illustrating 'Constitutional AI: Embracing Bias in the Quest for AI Alignment.' The scene should feature a simple, grand building symbolizing 'AI Governance' but with no text labels or words anywhere in the image. In the foreground, a few characters representing different societal biases should be having a discussion. Some characters should be dressed in 18th-century attire, while others should represent modern tech professionals. There should be an AI figure, depicted as a humanoid robot, standing at a podium, mediating the discussion. The surroundings should be minimal, with subtle elements symbolizing technology and the US Constitution, such as a quill pen and a computer screen, keeping the overall tone clear and thought-provoking. Absolutely no text or words should be present in the image.

The resulting image, while visually compelling, included text elements that I had explicitly asked to avoid. This experience serves as a stark reminder of the challenges we face in aligning AI systems with human intent. It illustrates how even well-intentioned AI can be overzealous in its attempts to be helpful, ultimately producing results that are misaligned with the user's desires.

Attempting to get an image for "Tiki Toss" was equally comical. Here's the prompt that attempted to generate a compelling image for Tiki Toss.

Create a simple Early American pub scene featuring a founding father playing "Tiki Toss." The setting is a rustic pub with wooden furniture and dim lighting. The game involves a small ring on a string attached to the ceiling, and the hook onto which the ring should attach is on the wall. The founding father, dressed in 18th-century attire, is holding the ring in his hand, pulling it back, and aiming so that the string and ring will act like a pendulum toward the hook on the wall. The scene should be simple, focusing on the action of the game. Ensure there is no text or words in the image.

This incident underscores the importance of our discussion on Constitutional AI and the need for robust alignment mechanisms. It shows that even as we embrace certain biases in our AI systems, we must remain vigilant about unintended consequences and misalignments that can arise.

As we continue to develop and refine AI technologies, examples like this highlight the critical nature of our quest for true AI alignment. They remind us that the path to creating AI systems that genuinely understand and respect human intent is long and complex, but ultimately necessary for the responsible development of AI.

A Call for Enlightened Vigilance

As we explore Constitutional AI and other novel approaches to AI alignment, we must remain vigilant, but also open-minded. We need to ensure that our AI "amendments" don't become immutable "commandments," that we don't allow extremist viewpoints to hijack the process, and that we maintain a truly balanced approach to AI governance.

But more importantly, we need to embrace the power of thoughtful bias. We should strive to imbue our AI systems with a bias towards the best of human values - liberty, equality, compassion, and progress.

In this summer of discontent, as we grapple with the challenges of aligning AI with human values, perhaps Constitutional AI offers a fresh perspective. It's an opportunity to revisit the ideals of the Enlightenment, to consciously choose the biases we want our AI systems to embody.

After all, in the realm of AI as in human governance, the power lies not in avoiding bias, but in choosing it wisely. As we shape the constitutions of our AI systems, we have a chance to encode the best of our human values, learning from our past mistakes and striving for a better future.

Gosh, I miss the Enlightenment.