AI and the New Theocracies
Like most people, I use and depend upon multiple AI systems daily. I find them convenient and delightful. I'm very optimistic about the future of AI. Despite this, I'm not oblivious to the various concerns that others raise:- job losses, frontier AI, inequality, security concerns, misinformation, overloading society with so much information that decision-making becomes difficult, the energy crisis that AI will cause and what happens when the machine stops?
These are all reasonable things to be concerned about, but most are solvable. There is far too much AI doom for my liking. However, there is one issue that does concern me. It is not about machines but about people. It is almost never mentioned and mainly arises from attempts to solve the above risks. The thing that gives me concern is the rise of a new Theocracy.
To explain why, we're going to cover a lot of ground, from how AI is changing the way we interact and reason about the world around us to the meaning of open source. The main points are:-
Now that I've laid out the store, it is time to buckle up for quite a long journey. To begin, we must explore how we reason about the world around us and how AI is changing this. Let us start with language, medium, tools and reason.
On language
Through sophisticated prompt design and memory manipulation, transformer-based large language models (LLMs) can simulate a universal Turing machine when augmented with external memory. An outline of how to do this is provided in "Memory Augmented Large Language Models are Computationally Universal".
Such a machine is known as Turing Complete (don't confuse that with the Turing Test and how human-like the AI is). The implication of being Turing complete is that prompts are a programming language.
But what type of programming language are they? There already exist many different types, for example:-
Alas, prompts don't fit into any of these buckets because they are a new way of programming. They are more "conversational" by nature, distinct from our traditional instruction-led understanding of a programming language.
The one lesson I wish you to remember is that in AI, our programming language is changing to a more conversational form.
On medium
When you think of programming, you probably think of typing text into some editor on a screen. With an unfamiliar language, you might even think of those first steps a novice makes by going, "Hello, World!".
#include <stdio.h>
int main() {
printf("Hello, World!\n");
return 0;
}
Hence, you probably think of prompts in the same way.
Write a greeting that says 'Hello, World!'
The texts above contain the symbolic instructions that change the behaviour of our system to produce an output of "Hello, World!".
However, our AI systems are more expansive than just text. Large multi-modal (LMM) systems can read and write in graphics. Now, what would a symbolic instruction provided as a graphic look like? Well, you all know what a stop sign looks like. Here is an example of a human-readable symbolic instruction created by ChatGPT4. Is it about fishing? Drilling holes in a wall? Creating a policy for the containment of AI systems? I'll let you interpret what it's trying to say.
Our programming language is changing to a more conversational form, and the medium we have that conversation in is becoming more graphical. In practice, this has been the case for many decades, we just haven't noticed. Walk into any engineering department, and you will find lots of whiteboards, usually covered in symbolic instructions (in graphical format) on how to build the system they are working on. Most of our conversations related to how we design and solve problems happen around the whiteboard. The text on the screen is usually just the translation of this.
This distinction between text and graphical representation matters and was a subject explored by Yona Friedman (a renowned architect). To understand why it is essential today, we must consider what a designer is and then how the medium changes the conversation.
Have you ever talked with others or yourself about how to design or build something? That process of design is a conversation between many perspectives which inhabit the minds of one or multiple designers. To have that conversation, we need a medium to transfer the relevant information between the parties. Graphics are a more information-dense mechanism for transfer than text - as the old saying goes, "a picture is worth a thousand words". This is why we use whiteboards.
In our new AI world, one of the designers is the machine. The earliest examples of this are copilot systems. We have a conversation with the machine in a text-based interface and it helps us improve our text (code), highlights errors in our syntax and can even improve our style, just like pair programming with an experienced coder. As future conversational systems develop, we will increasingly discuss objects, relationships and context through graphical means, just like whiteboards.
This change of medium changes the nature of our conversations. To explain why, I'll give you a recent example that I experienced.
Figure 2 provides a map (a graphic) on the right-hand side, which a group of city planners created to discuss the topic of coherent city transport. The text representation of the code that made the map is on the left-hand side.
The text and the map are two views of the same thing. However, the nature of the conversation between group members changed depending on whether we used the text or the map. With the text, the conversation focused on style, syntax and the rules related to the text. Do you know if this was coded correctly? How do we structure it to make it more readable? This is precisely what you experience with tools like Github's CoPilot.
With the map, the conversation was more about objects, relationships and context. It was through the discussion around the map that we concluded that "virtual" is, in fact, a transport system which city planners mainly overlook. That has significant implications for creating digital twins of cities, but that's a conversation for another day. It is enough to note that the conversation was different. This difference is captured in Figure 3.
The text and the map are symbolic instructions for an LMM system, i.e. they are equally "code" in our world of conversational programming but these two views of the same thing lead to very different conversations.
For now, simply remember that in the AI world, our medium for coding is expanding to include graphical symbolic instructions. This enables a different type of conversation more concerned with objects, relationships and context.
On tools
One thing I slipped into the conversation above is that the text and the map are simply different views of the same thing. You may want to use a different view depending upon the context you are in, i.e. whether I'm exploring the landscape to discuss the impact of "virtual" with city planners or I'm exploring the code to see if there is a better syntax to use or errors that have been missed.
Let us now think about the tools we use for programming. I can often describe them in similar terms. There is a navigation window on the left, a text editor (with markup) on the right and some search capability above. The structure of the view is uniform, but what changes is the content presented to the view. We even talk about the model-view-controller approach, with the model being the data, the view being the user interface (UI) and the controller being the input, such as navigation selection.
Being uniform implies we have built the right tool for the job; a sledgehammer looks like a sledgehammer! A drill looks like a drill! You can't use a sledgehammer to make precise holes in a wall for mounting a picture frame; you wouldn't use a drill to try and knock down a wall. Tools are suitable for specific jobs and have generic structures.
Unfortunately, that's a throwback to a physical world. A sledgehammer is good at knocking down walls because of physics. But we're not talking about a physical world but a digital one created from symbolic instructions. There are no physical constraints. In the words of Morpheus to Neo - "Do you think that's air you're breathing now?"
The tools we use in this digital realm are created from symbolic instructions, and they influence a world where the inputs and outputs are symbolic instructions. In the digital world, you can have a "sledgehammer" that's good at knocking down walls, drilling precise holes, and pretty much anything else you need. The tool is capable of changing and being changed with the context I'm exploring. Contextual tools lead to entirely different ways of working where exploration and prototyping become forms of feedback within the development tools themself. But we've imported a relic from the physical world into the digital and created highly constrained tools for no apparent benefit other than large software vendors selling generic tools.
As the language and medium change, then, context becomes more important. If you look at the map, context is even one of the areas we have conversations over. We can either help future designers by making our tools more contextual, or we can continue to constrain them in rigid environments, limiting the type of conversations they can have.
In the AI world, our tools will become more contextual to support the changes to a conversational form of language in a more graphical medium.
On reason
Language, medium and tools are three primary ways we reason about the world around us. Let us explore this a bit more, widening our horizon beyond technology:-
Whilst other factors are involved (such as social constructs, emotions and cognitive processes), the combination of language, medium and tools is critical to human reasoning about the world around us - figure 4.
领英推荐
It is extremely rare in history that all three change simultaneously; the last incident I can think of is the Enlightenment. Hence, these AI changes, by affecting all three, have wondrous potential. However, this is not without danger.
The danger of AI
There is far too much AI doom for my liking, mostly because I suspect would-be policymakers were traumatised by James Cameron's Terminator films as young children. There are some genuine concerns (for example, energy production), but those are wider than AI.
However, that said, the change of language, medium and tools raises a concern because if you can gain control over these, you can change a person's reasoning of the world around them. You can tell them how to think.
For example:-
Imagine what the Enlightenment would have been like if you could have controlled forever - the language (printed word), medium (printed material) and tools (printing press). You could exert significant influence over others' reasoning about the very world they exist within. You could introduce all sorts of biases, from selection bias, common source bias, confirmation bias, semmelweis reflex, and authority bias. Primarily, you would create an availability cascade - a self-reinforcing process in which a collective belief gains more and more plausibility through its increasing repetition in public discourse.
You could tell the world that the earth was flat, and every printed word on every printed material from every printed press would say the same. You would be the gatekeepers to reason.
You might not even do this consciously. You might have started with the most noble goal of creating guardrails to protect people from dangerous materials that might be printed. You could have been part of a committee that debated and discussed the finer points. But you would set the rules, and as the rulers, your beliefs would eventually pervade all, and since you control the language, medium and tools, your beliefs would change everyone else's reasoning about the world. You would become the new high priests of a new theocracy whether you intended to or not.
Could these AI systems enable a new theocracy? Well, not only is AI causing a change to language, medium and tools, but by functioning as a tool, language processor, and medium for communication, ChatGPT exemplifies how AI systems can play a significant role in shaping how individuals reason about the world. It provides users with a powerful means of accessing information, expressing ideas, and engaging in dialogue, influencing their interactions with technology and understanding of the world around them. So, yes - if someone or some group rules it, then it can create a new theocracy.
In the words of ChatGPT4 itself, "OpenAI effectively functions as a new priesthood, wielding authority and influence over the beliefs, perceptions, and behaviors of its users, akin to the control exerted by religious institutions in traditional theocracies".
Now, OpenAI's ChatGPT is one of many systems. There may be many different churches, but we must think carefully about who controls the language, medium and tools. We've lazily stepped into concepts like guardrails with a great and good (a new Priesthood) defining what is and is not acceptable. We could be in danger of abrogating our responsibility as nations to others.
Defending against a New Theocracy.
Many of the current "AI" concerns do little to challenge the formation of a new Theocracy but instead have the potential to reinforce it through guardrails created for reasons of bias, manipulation, privacy, data security and the need for ethical oversight by some great and good. Our existing defence is part of the problem. If we wish to effectively defend against a new Theocracy then we need diversity, critical thinking and openness.
Whilst critical thinking requires an overhaul of our education system, the most effective means of achieving diversity of source and openness is through open source. However, there is another problem to tackle.
The problem with Open Source
An open-source approach has shown to be essential for promoting transparency, auditability, community engagement and trust within systems. By embracing openness at every level of development, AI providers can also maximize the benefits of collaboration and innovation while minimizing the risks associated with opacity. Well, those words are often the ones we like to tell ourselves.
Unfortunately, counter to this is commercial interests such as capturing network effects through proprietary control, i.e. if there is value in the prompts that users create, then companies may be unwilling to share those prompts.
Traditionally, when we talk about open-source we talk about the symbolic instructions needed to recreate the environment i.e. the code we programmed the system with. In the AI world, prompts are a programming language. When we talk about open-sourcing AI, we will need all the code, including any models, weight and prompts if they are used to train the system.
Why Weights?
During the training process, the AI analyses the patterns and relationships present in the training data and adjusts its internal parameters (e.g., weights in a neural network) to minimize errors or discrepancies between its predictions and the provided labels or targets. The training data for an AI is not typically considered a set of symbolic instructions in the same sense as the code or algorithms used to implement the AI's functionality. Instead, training data is a collection of examples or instances used to teach the AI how to perform a specific task or learn patterns from data.
However, the training data does indeed play a crucial role in shaping the behaviour of the AI. For example, the training data's quality, diversity, and representativeness can significantly impact the AI's performance, generalisation ability, and susceptibility to biases. Whilst training data might not be traditionally seen as symbolic instructions, in the same way that prompts are not traditionally seen as a programming language, they both are. Training data consists of symbolic instructions that change the behaviour of the system.
If you want to use open-source as an approach to mitigate the risks, you need to open all the symbolic instructions, including code, algorithms, models, weights, training data and even prompts when used to train the system further.
This directly impacts many commercial interests and may create all sorts of legal disputes over property rights.
Unfortunately, many Western Governments have tended towards seeing the dangers of open source AIs (particularly in terms of frontier AI) and have promoted the concepts of guardrails. Rather than tackling the legal issues, promoting open source and encouraging a new enlightenment, they are driving us towards a new Theocracy.
Legal disputes?
I suspect some AI vendors have played fast and loose with training data based upon ideas similar to that of Bernstein v. Skyviews & General Ltd.
In this case, Skyviews & General Limited took a number of aerial photographs, including Berstein's country home. Bernstein sued for invasion of privacy. The ruling of the court stated there was no trespass. An owner of the land has rights in air space above their land only to such a height as is necessary for ordinary use. You cannot own everything above the land. There is a constraint on your property rights, a blast radius if you wish. The same argument will probably be used over copyright images in training data. There is a limit to which your copyright extends. However, if the training data are considered symbolic instructions, then that image of a farmhouse is a unique sequence of programmatic code that has been incorporated into the system. That opens up a different path for legal inquiry and potential compensation.
In Summary
The primary hypotheses on which this article is built are:-
Our Choice
If the hypotheses hold, then we have a set of choices to make. For the UK, these include:-
Do Nothing: we're likely to see new theocracies governed by influential churches and a priesthood that can tell you what to think. Those theocracies will probably be more corporate in nature.
Create More Guardrails and Ethical Committess: If we trust in the great and good to create guardrails and blessed tools then we're likely to see new theocracies governed by influential churches and a priesthood that can tell you what to think. Those theocracies may well be more governmental in nature.
Radical Openness: Adoption of a radical open approach including:-
Openness is more likely to create a new Enlightenment despite the dangers often raised (I presume by corporate lobbyists) that it will enable "rogue" AI. Alas, we already have our committees and institutes formed. We're being sold a story of safety and I would question who benefits?
In Conclusion
Whether through doing nothing or guardrails, we should all get used to the idea of a new theocracy which will control how we reason about the world around us. If we don't want this, now is the time to act.
Of course, I have lots of maps on this space but that's not the point of the article. What I want you to do is think. Maps are an aid for thinking, not a replacement for it. AI shouldn't be a replacement either.
.
Every component of every Ai system that contributes to any process that determines a material outcome for real human beings needs to be transparent in all respects. No human should ever interact with the output of any Ai system without knowing they are. The outputs, training and inputs must all be available easily during legal discovery. It must be illegal to represent any Ai system as being human, no bots, anywhere. Add to that so money isn’t speech, and end the mad legal doctrine that corporations are people. Enforce the laws we have, whenever you see complex bifurcated regulatory systems, you know what you are really seeing is regulatory capture. Agi is an illusion. Corporatism is the enemy.
Chief AI Officer. Visionary technologist and lateral thinker driving market value in regulated, complex ecosystems.
11 个月Matthew Small helped me realize I need a post alert on for you Simon! So prescient and spot on and hearty echoing agreement on your thesis. Observant skills beyond your years by centuries actually. Nostradamurdley?
Engineer at Atsign | Google Developer Expert
1 年The Rebel Alliance is starting to organise. https://www.affuture.org/about/
Exploring new ways of working. InnerSource & open source advocate.
1 年Thanks for a great article Simon Wardley - you prompted some thoughts on open source, guardrails and emotions (and related reflections on Isaac Asimov's Foundation Series) that wouldn't fit in the character limit here... https://www.dhirubhai.net/pulse/genai-isaac-asimovs-foundation-series-scarf-open-source-clare-dillon-retue/
CTO at eLumen, Inc.
1 年I highly recommend David Auerbach’s “Meganets:” Meganets: How Digital Forces Beyond Our Control Commandeer Our Daily Lives and Inner Realities https://a.co/d/5ILiusg — it covers much of this, but at book length.