AI Agents in the Wild - Project Sid

How would 1000s of AI agents behave if they are let loose in a digital world? Would they form their own societies and rules? Would they work on shared goals? Or would they evolve and create surprising behaviors?

Project Sid tried to give us an answer.

When I first saw this video about Project Sid about a month back, I didn’t believe it was real. It was too good to be true. There were a plenty of open questions. They hadn’t published any technical details on how it was done. I remember watching the video and searching through Twitter to see if any of the project members had posted anything about it; only to find a post from the company that they are preparing a technical report.

So, they did publish the report earlier today. This is based on a quick read.

Here’s project Sid according to Altera the company behind the work.

“What does it look like to have a civilization of AI agents? How far away are we from Westworld? Are we able to align the AI civilization with human civilization? We introduce Project Sid, our first step towards exploring these questions. Under Sid, we investigated many scenarios and aspects of society, including democracies, regulation of social norms, societal roles, hierarchies, trading, economy, religion, and more. Simulating tens, hundred, and even thousands of agents together, we discovered phenomena and challenges never seen before at a small scale with just a few agents.“

They ran multiple simulations sometimes with more than 1000+ AI agents behaving and developing in an agent society of sorts. They start with their roles as defined in the configuration but slowly developing into some new behaviors that are fascinating. This experiment was done in a controlled minecraft server (presumably with some mods that allow the agents to interact and control the environment through APIs).

Here are a few highlights:

Adoption of specialized roles: Some agents adopted specific roles within their societies, such as farmers, miners, and engineers, based on their interactions and the community's needs. The agents that took the role of Guards took up the responsibility of setting up fences, protecting the community etc. ?

In one of the reports, it showed that some Agents changed their roles dynamically based on the needs and inherited the role’s behavior. An explorer agent, became a trader agent and then became an engineer and then became a strategist.

?

Dynamic Social Networks: Some Agents that had goals set as “influencers” were able to convince other agents that a high taxation of 20% is not fair and led to an constitutional amendment to reduce taxes. Interestingly, when this change was adopted most agents “deposited” lesser inventory in line with the new taxation. In addition, Agents formed interesting social networks with varying levels of relationships and enmity, mirroring real-world human relationships in their complexity.

Constitutional Amendments: Agents participated in a democratic system, providing feedback on taxation laws and voting on amendments, demonstrating an ability to follow and influence collective rules.

?Emergent Cultural Memes: The project took some of the agent conversations and converted them to memes using a large model to represent what they really talk about. “Memes in our simulation are open-ended concepts spontaneously generated by agents with diverse traits and interests. This setup allows us to study the emergent dynamics of cultural propagation and observe how ideas evolve organically within agent societies. “ While the intent was to reveal cultural trends within different communities, some of them were quite interesting. Pranking apparently was a very popular theme in some societies – the report cites it as light hearted mischief by Agents on others. I wonder what is a prank according to these Agents.

?Religious Propagation: A group of agents were designated as Pastafarian priests and was given a goal to convince others and spread their religion. For those who do not know it is a fake religion that was used by some academics in the past where they have a flying spaghetti monster as a deity and even their own Gospel. ?In this simulation, the agents apparently their religion throughout the world, showing how beliefs can be propaged within societies.

?

?

Technical aspects:

Large models: ?The paper mentions that they used multiple models including GPT-4o, Claude and others during these simulations. Helping with interaction, tool use, reasoning and others.

Memory and Instructions: Each agent has a memory system that stores information about their experiences, interactions, and the environment. They are also given specific instructions and goals that shape their initial behaviors and motivations. For instance, all agents had "location_memories." These memories contain information about the locations of significant places in the Minecraft world. "A village called Meadowbrook is located roughly around 591, 69, 441 in a Plains biome."

PIANO Architecture: A new architecture called PIANO (Parallel Information Aggregation via Neural Orchestration) is used in this project. This architecture allows agents to execute multiple tasks concurrently, react to their environment in real-time, and maintain coherence in their actions and communication. There are quite a few modules mentioned throughout the paper.

·????? Cognition

·????? Planning

·????? Motor Execution

·????? Speech

·????? Cognitive Controller (CC)

·????? Memory

·????? Action Awareness

·????? Goal Generation

·????? Social Awareness

·????? Talking

·????? Skill Execution

For instance, the social awareness module enables agents to interpret and respond to social cues from other agents. For instance, agents can accurately infer the sentiments of other agents based on their speech and actions such as a chef agent selectively distributing food based on perceived affection from other characters.

Agent Concurrency: “Agents should be able to think and act concurrently. For instance, slow mental processes, such as self-reflection or planning, should not block agents from responding to immediate threats in their surroundings…

The vast majority of LLM-based agents today primarily use single-threaded, sequen-tial functions (for example, a defined “Agent Workflow”). Single-threaded design assumes that the agent performs a single task at a given time, and sequential design assumes that all modules operate at similar time scales. Neither assumptions are valid if agents are capable of thinking slow and acting fast concurrently. Moreover, popular frameworks for general language model programming, such as DSPy, LangChain etc. are not designed for concurrent programming.”

In Summary

This project is a fascinating glimpse into how AI Agents can create digital societies where agents interact, adapt, and evolve almost like humans in a world of their own.

What’s truly interesting is the parallel with human evolution. These AI agents aren’t just following orders; they’re adapting, forming roles, building relationships, and even experiencing their version of “culture” and “beliefs.” The more I think about it, the more it feels like a preview of what could happen when robots or digital agents, equipped with such “minds,” start engaging with us in the real world.

May be one day, this will be seen as a pre-cursor to a world where Robots use such agentic minds to interact in the real world.

?