Part 5 - Rules of Engagement: Establishing Governance for Multi-Agent Generative Systems

In parts 1 to 4 of the series, we explored the potential of Multi-Agent Generative Systems (MAGS) to transform industrial processes. We examined how these systems can enhance decision-making, improve efficiency, and adapt to complex operational environments. As we move forward with implementing MAGS for industrial settings, it's essential to address a fundamental aspect of their deployment: the rules that govern their behavior.?

In this fifth article in the series, Dr. Zoran Milosevic and I will focus on the concept of "Rules of Engagement" for MAGS. These rules serve as the foundation for the responsible and effective use of AI agents in industrial environments. They ensure that our technological advancements align with organizational goals, legal requirements, and ethical standards.?

The need for clear rules becomes apparent when we consider the autonomous nature of MAGS. These systems can make decisions and take actions with minimal human intervention. While this autonomy offers significant benefits, it also presents risks if not properly managed. Establishing robust rules of engagement helps mitigate these risks and builds trust in the technology.?

Our discussion will cover various types of rules, including regulatory, organizational, professional, legal, and ethical guidelines. We'll explore how these rules apply to different entities within a MAGS ecosystem, including human operators, organizations, and AI agents themselves.?

We'll also examine practical approaches to implementing these rules. This includes standards-based enterprise modeling and other structuring methods that can define intentions, trust relationships, and interactions between agents and humans. We will explain how these Rules of Engagement are areimplemented in XMPro MAGS.?

Let's begin by examining the fundamental concepts of rules and their importance in complex systems like MAGS.?

Engagement Rules: Why??

Complex systems consist of diverse actors, each with their own goals, as reflected in their intentions and behavioral preferences. These actors can engage in interactions and collaborations within existing structures, such as organizations, where they accept obligations or commitments associated with their defined roles. They can also form new partnerships with other actors based on mutual agreements.

While internal rules guide the behavior of actors as independent entities and their involvement in collaborations, they must also adhere to external regulations. Both internal and external rules contribute to maintaining system stability, predictability, and trust in an uncertain environment.

Our earlier articles demonstrate how AI agents are becoming an important element of complex systems, addressing the resource scarcity, data complexity and unlocking of new economic values. ?

AI agents, engaged by humans or organizations, are a type of actor whose behavior can simulate or mimic that of humans. However, ultimate accountability lies with the parties that have legal responsibilities, such as human creators or organizational owners. While AI agents may delegate tasks to other agents, the ultimate responsibility remains with the human or organization. The importance of rules in this context is to clearly define responsibility and accountability across both human and AI agent entities.?

We use the term Rules of Engagement to encompass the various types of rules needed to govern the behavior of actors in complex systems. These rules can be used to define obligations, permissions, and prohibitions for actors involved in collaborative activities. They can originate from internal governance arrangements or from external rules.

The latter include regulatory compliance, legal requirements, ethical considerations, professional conduct, engineering and security, safety and risk management, resource management, environmental responsibility, and more.

Engagement Rules: What??

?When considering computable approaches for supporting engagement rule expressions, the above considerations suggest a need to express:

• Organizational context that defines rules, whether the context captures a regulatory domain that specifies controlling rules over members of the domain (both individual actors and organisational structures), or a collaboration structure whose objective drives the rules of collaboration.
• Legal entities that have their own independent life and identity, regardless of their participation in any collaboration
• Collaboration structures that can define templates for interactions; these allow for multiple instantiations, such as different parties filling roles in such collaboration structures in different times.
• Organizational or legal policies that cover constraints over behavior of both the actors (parties and AI agents) and collaboration structures, providing guardrails over their autonomous behavior.
• Range of organisational or legal policy types, including simpler rules associated with obligations, permissions and prohibitions, but also their derivation, needed for more complex ?expressions of accountability and responsibility (see Figure 2).
• Dynamics of the obligations, permissions and prohibitions as a result of authorisation or delegation of services or responsibilities across actors, while ensuring clear traceability of responsibility across the actors
• Agentic behavior to define possible actions of actors over time, reflecting their objectives while also responding to the actions of others.
• System policies that can support the variability of system design over time and thus facilitate system evolutions, including the change of organisational or regulative policies.

Engagement rules: How??

?When building complex systems, the engagement rules should be articulated in a computer interpretable style, to ensure consistency, interoperability and evolvability of the systems, as the operating environment rules change, including the availability of new technologies.

Such computable expressions in turn requires formal models to ensure that the computable language can be interpreted using suitable tools, including those that check consistency and traceability from business requirements to implementation, in support of conformance and compliance checking.?

One practical approach to implementing such rules is based on formal models embedded in the concepts specifically developed to design and implement large or complex systems, namely the ISO/ITU-T/IEC open distributed processing systems (ODP) standard [1]. One component of the standard, the ODP Enterprise Language provides [2] much of the semantics needed semantic foundation for describing rules of engagement.??

(* adapted from an upcoming conference publication by Zoran Milosevic )

For example, the ODP concept of party, can be used to model actors thar have broad set of responsibilities derived from some social or legal framework, and these can be natural persons or organisations.?

Parties may engage automated agents, such as AI agents discussed previously, to perform actions on their behalf, and this involves delegation of their responsibility to address resource issues, either to do with capability specialisation or scalability reasons.?? The aim here is to be able to trace the way that the rights and responsibilities of the parties are linked to the individual system actions and their consequences, which is key to ensuring building responsible digital twin systems.??

Further, the ODP concept of community is useful to define the collaboration structures introduced earlier, which serve as templates for interactions of actors participating in collaborations. A community is essentially a contract specifying how actors can participate in different community roles to fulfil their objectives, and that of the community’s objective. The contract is expressed in terms of obligations, permissions or prohibitionspolicies that apply to the actors fulfilling these roles. ?

These policies, often referred as deontic policies, are intended to capture the rules of the community contract, including those that propagate to the community, from the outer regulatory environment. Note that the term "Deontic" has roots in philosophy and logic, specifically in the study of obligation, permission, and prohibition and in recent AI developments they are employed to design systems that adhere to rules and regulations, such as access control mechanisms, legal frameworks, and ethical guidelines for AI [3].??

The ODP-EL standard provides a practical implementation path for handling these deontic and accountability concepts, through encapsulating them within objects that can be handed over across the actors in the system, referred to as deontic tokens [2]. Here, the holders of these tokens, are constrained? by the nature of policies encapsulated within them. This is similar mechanism as used in many recent security policy approaches such as OAuth 2.0 access tokens, although the constraints here are broader and apply to any holder’s actions rather than to narrower, data access actions, in access control policies.

Further, deontic tokens can be associated with some important actions performed by actors, when they result in changes of obligations, permissions or prohibitions associated with the actors.?

These actions are referred to as speech acts, introduced following the linguistic concepts of speech act, and they can be used to express how certain actions such as requests, orders or promises, change the state of world, from the perspective of obligations or permissions of actors involved. They are powerful way of expressing chain of responsibility across parties and their AI agents.

This foundational model for expressing engagement rules as constraints on expected behaviour of agents, also serves as a basis for the inclusion of related modelling concepts/formalism and techniques needed for analysing and reasoning about system properties related to AI agent technologies.

One such technique, referred to as Promise Theory [4], is suitable for describing intention of AI agents and how these can be related to their objectives, future actions, and decisions, and thus the deontic formalism mentioned above. Such decisions can be influenced by agents own objectives and commitments, while allowing for potential choices associated with expected behaviour and trust in other agents with which they can interact and collaborate.??

How do we implement the Rules of Engagement in XMPro MAGS??

XMPro's Multi-Agent Generative System (MAGS) uses a structured approach to implement Rules of Engagement for AI agents.. This framework ensures AI agents operate within defined boundaries and follow organizational policies. It also helps agents make decisions that align with ethical standards and business objectives, which is crucial in complex industrial settings.?

Here is an example from the oil and gas industry. A major refinery uses XMPro MAGS to manage its operations, from production planning to maintenance scheduling and safety monitoring. The system includes various AI agents responsible for different aspects of the refinery's operations, each needing to work in harmony with others while adhering to strict industry regulations.?

Agent Profile System Prompts serve as the foundation for establishing overarching rules in XMPro MAGS. These prompts include:?

Organizational rules: These are high-level directives that reflect the company's values and policies. For example:?

"Prioritize safety in all operations" 
"Comply with environmental regulations at all times" 
"Optimize resource utilization to minimize waste" 
"Maintain product quality standards across all production lines"         

Deontic rules: These specify what agents must, may, or must not do in various situations. Examples include:?

"Agents must report any safety anomalies immediately" 
"Production agents may adjust output levels within specified ranges" 
"Maintenance agents must not schedule repairs during peak production hours unless critical"         

To implement these rules flexibly and dynamically, XMPro MAGS uses a system of Deontic Tokens. These tokens are based on concepts from the ISO/ITU-T/IEC open distributed processing systems (ODP) standard, ensuring a standardized approach to rule implementation. Each token represents a specific rule, including its type, the subject it applies to, and the conditions for its application.?

The XMPro MAGS Agent Memory Cycle integrates these tokens at every stage of an agent's decision-making process:?

1. Observation: Tokens guide what information agents can access and how they interpret it. For instance, a safety monitoring agent might be authorized to access real-time sensor data from all parts of the refinery.?
2. Reflection: Agents consider applicable tokens when evaluating past actions and forming new insights. This allows them to learn from experience while staying within defined boundaries.?
3. Planning: Token-based rules influence the creation and prioritization of action plans. A production planning agent might adjust its scheduling strategy based on recent performance data and current operational constraints.?
4. Execution: Before any action, agents verify compliance with relevant tokens. This ensures that even in dynamic situations, all actions align with organizational rules and regulations.?

This approach ensures that Rules of Engagement actively shape agent behavior in real-time. It allows the refinery to maintain safe, efficient, and compliant operations while adapting to changing conditions. For example, system administrators can quickly update the relevant tokens through the Agent Profile if a new environmental regulation is introduced. All affected agents will immediately adjust their behavior to comply with the new rule, without reprograming each agent individually.?

By implementing Rules of Engagement through Agent Profile System Prompts and Deontic Tokens, XMPro MAGS balances consistent agent behavior with operational flexibility. This method offers several benefits:?

• Ensures AI systems remain aligned with business goals and regulatory requirements?

• Provides a clear audit trail of decision-making, crucial in highly regulated industries?

• Allows for quick adaptation to changing rules or conditions?

• Maintains consistent behavior across different agents and processes??

• Enables fine-grained control over agent permissions and restrictions?

• Facilitates easier updates to system-wide policies without extensive recoding?

The system also supports hierarchical rule structures, allowing for both broad, organization-wide rules and specific, task-level rules. This hierarchical approach ensures that agents can handle complex, nuanced situations while still adhering to overarching principles.??

In the refinery example, this might look as follows:?

• Top-level rules about safety and environmental compliance?

• Department-level rules about production targets and quality standards?

• Task-specific rules for individual processes or equipment?

This layered approach to Rules of Engagement helps organizations implement AI systems that are both powerful and controllable, suitable for complex industrial environments like oil and gas refineries. It provides the flexibility needed to optimize operations while maintaining the strict control necessary in high-risk, highly regulated industries.??

How to get started with RoE and MAGS

As we've explored in this article, implementing Rules of Engagement is essential for the successful deployment of Multi-Agent Generative Systems (MAGS) in industrial environments. These rules, when properly designed and executed, help organizations balance AI autonomy with necessary controls. XMPro's approach, using the Deontic Token System, offers a practical way to manage AI agent behavior while maintaining operational flexibility. This method allows companies to adapt quickly to changing regulations and business needs without compromising on safety or compliance.

For senior managers and technical leaders considering MAGS for their operations, understanding these concepts is crucial. The potential benefits of MAGS are significant, but so are the challenges of implementation. Every organization will have unique requirements for their Rules of Engagement, based on their industry, risk profile, and strategic goals.

If you're interested in exploring how Rules of Engagement and Deontic systems could work for your company, we encourage you to take action. Dr. Zoran Milosevic and Pieter van Schalkwyk are available for in-depth discussions about applying these concepts to your specific situation. Their experience can help you navigate the complexities of AI governance in industrial settings. To arrange a consultation, please reach out through our website or LinkedIn profiles. We look forward to helping you shape the future of your operations with responsible AI implementation.

References

[1] Linington, P.F., Milosevic, Z., Tanaka, A., Vallecillo, A.: Building Enterprise Sys- tems with ODP: An Introduction to Open Distributed Processing, 1st Edition. Chap- man&Hall/CRC Innovations in Software Engineering and Software Development (2011)

[2] ISO/IEC IS 15414, Information Technology - Open Distributed Processing - Enterprise Language 3rd edn, 2015. Also published as ITU-T Recommendation X.911.

[3] Z. Milosevic, Ethics in Digital Health: a deontic accountability framework, Proceeding of EDOC2019 conference, https://ieeexplore.ieee.org/abstract/document/8945023

[4] Burgess, M., & Bergstra, J. A. (2014). Promise theory: Principles and applications. Springer.