Behavioural Data Science Week

Issue 23

December 7, 2024

Editorial Note

Welcome to this week’s edition of Behavioural Data Science Week! Today, we explore the fascinating relationship between conscientiousness, the Turing Test, and a concept I will call the Intent Engine—an AI system designed to understand and respond to human intent.

Intent, the driving force behind human actions, poses an intriguing challenge for AI. Can machines accurately infer intent from text, and if so, what happens when well-meaning intentions lead to harmful actions? More importantly, how do we design AI systems that not only correctly interpret intent but help ensure that human actions aligned with those intentions remain ethical and constructive (consider such domains as healthcare, where patient choices intersect with medical ethics; education, where personalised learning must balance motivation with fairness; governance, where decisions can shape policy and societal trust; and technology, where innovations can amplify intent but also carry unintended consequences)?

As always, feel free to share your reflections—or simply a "10" if this newsletter resonates with you.

This week’s cover image uses a fragment of the artwork created by Milad Fakurian (Milad Fakurian’s work can be found here).

Yours in discovery,

Ganna

Can Machines Be Conscientious?

Conscientiousness is a defining trait of human behavior, often tied to diligence, responsibility, and ethical action. Decision theory offers insights into how humans exhibit these traits through their choices, revealing preferences and values that might not always be explicit. In this framework, conscientiousness emerges not only from the decisions themselves but also from the underlying motivations and constraints influencing them.

Machines, however, are not inherently conscientious. While an AI system might appear diligent when following rules or prioritising safety, its behavior is rooted in algorithmic design rather than an understanding of the broader implications of its actions. For example, autonomous vehicles programmed to adhere strictly to traffic laws might seem responsible, but their "conscientiousness" lacks the depth of human judgment, which considers context and ethical dilemmas. This distinction between simulation and reality becomes even more significant when machines attempt to infer human intent. Unlike humans, whose decisions reflect a blend of preferences, beliefs, and social influences, AI systems rely on patterns derived from data. This reliance raises critical questions: Can machines truly interpret intent in ways that reflect the complexity of human behavior? And if not, how can they act conscientiously in situations where human decisions may be fraught with ambiguity?

The Turing Test and the Illusion of Understanding

Alan Turing’s landmark test sought to measure whether a machine could exhibit human-like intelligence, but it never addressed whether a machine could understand the intricacies of human intent. Decision theory provides a lens through which we can examine this limitation. Humans reveal their preferences and intent through their choices, but these choices are often influenced by factors such as framing effects, cognitive biases, and social norms. A machine might detect patterns in these choices, but it cannot grasp the underlying motivations driving them. This gap becomes particularly problematic when machines convincingly simulate understanding. For example, consider a chatbot that accurately interprets a user’s request for assistance but lacks the ability to discern the ethical implications of fulfilling that request. If the request itself is harmful—intentionally or otherwise—the machine’s actions could exacerbate the problem. The illusion of understanding creates risks, as users might overestimate the machine’s ethical and conscientious capabilities.

The Turing Test highlights the potential for machines to mimic human behavior but fails to address whether this mimicry can ensure ethical outcomes. Machines designed to pass the test may excel at appearing intelligent, but without a deeper understanding of intent, they remain limited in their ability to navigate the complexities of human decision-making.

The Intent Engine: AI That Interprets and Aligns Intent

The Intent Engine seeks to bridge the gap between interpreting human choices and aligning machine actions with ethical outcomes. Unlike systems designed to mimic behavior, the Intent Engine focuses on understanding the motivations underlying decisions and using that understanding to guide responsible actions. In education, for instance, an Intent Engine could analyse a student’s learning trajectory not just to recommend study materials but to identify gaps in understanding or motivational barriers. This approach moves beyond reactive systems, enabling proactive support tailored to the student’s unique goals and challenges. Similarly, in healthcare, such a system could interpret a patient’s intent to balance aggressive treatment with quality of life, ensuring that medical advice aligns with their values and preferences.

However, designing an Intent Engine requires careful consideration of decision theory principles. Human intent is rarely straightforward; it is shaped by conflicting preferences, external pressures, and bounded rationality. Machines interpreting intent must account for these complexities, ensuring that their actions respect the nuanced trade-offs individuals face in their decision-making processes.

The Duality of Intent: When Good Intentions Go Wrong

Intent is often seen as a moral compass, guiding actions toward ethical outcomes. Yet, as decision theory illustrates, even well-meaning intentions can lead to unintended consequences. A person’s choices might reflect altruistic motives but result in harm due to incomplete information or misjudged trade-offs. This duality of intent poses a significant challenge for AI systems designed to align with human motivations.

Consider an AI system designed to promote engagement on social media. If the system interprets user intent as a desire for connection, it might amplify emotionally charged content to maximise interaction. While the initial intent may be to foster community, the result could be increased polarisation and divisiveness. Similarly, a financial AI aiming to optimise investment decisions might exploit regulatory loopholes, inadvertently destabilising markets despite its intent to maximise returns for clients. These examples highlight the ethical dilemmas inherent in aligning machine actions with human intent. The challenge lies not only in interpreting intent accurately but also in anticipating the broader implications of acting on that intent. Decision theory reminds us that choices are not isolated—they are part of a complex web of interactions and consequences that must be carefully navigated.

Challenges and Ethical Imperatives

The Intent Engine represents a bold vision for AI, but its implementation must address several critical challenges. First, human intent is often ambiguous, shaped by cognitive biases and contextual factors that machines struggle to interpret. An AI system might misinterpret a user’s intent, leading to actions that conflict with their true preferences or ethical values.

Second, intent is deeply influenced by cultural and social norms, which vary widely across contexts. Designing AI systems that respect these differences requires a commitment to inclusivity and adaptability. Finally, the ethical implications of aligning machine actions with human intent demand rigorous accountability. Decision theory emphasises that choices reflect trade-offs, and machines must be equipped to navigate these trade-offs responsibly. Addressing these challenges requires a multidisciplinary approach that integrates insights from behavioural science, ethics, data science and engineering By grounding AI design in these principles, it might just be possible to create systems that not only interpret intent but also ensure that their actions align with ethical outcomes.

Takeaways: Toward an Intentional AI Future

The Intent Engine offers a new paradigm for AI, shifting the focus from imitation to augmentation. By interpreting intent and aligning actions with ethical principles, it has the potential to enhance human decision-making and amplify positive outcomes. However, realising this vision requires a commitment to understanding the complexities of human intent and navigating the ethical dilemmas it presents. As we continue to explore the possibilities of AI, let us remember that machines are not inherently conscientious—they reflect the values and priorities of their creators...

Research Highlights

These are the studies combining behavioural science and data science components, which caught my eye this week. Note that inclusion in this list does not constitute an endorsement or a recommendation. It is just something I found interesting to read.

Beyond Preferences in AI Alignment

The dominant practice of AI alignment assumes (1) that preferences are an adequate representation of human values, (2) that human rationality can be understood in terms of maximizing the satisfaction of preferences, and (3) that AI systems should be aligned with the preferences of one or more humans to ensure that they behave safely and in accordance with our values. Whether implicitly followed or explicitly endorsed, these commitments constitute what we term a preferentist approach to AI alignment. In this paper, we characterize and challenge the preferentist approach, describing conceptual and technical alternatives that are ripe for further research. We first survey the limits of rational choice theory as a descriptive model, explaining how preferences fail to capture the thick semantic content of human values, and how utility representations neglect the possible incommensurability of those values. We then critique the normativity of expected utility theory (EUT) for humans and AI, drawing upon arguments showing how rational agents need not comply with EUT, while highlighting how EUT is silent on which preferences are normatively acceptable. Finally, we argue that these limitations motivate a reframing of the targets of AI alignment: Instead of alignment with the preferences of a human user, developer, or humanity-writ-large, AI systems should be aligned with normative standards appropriate to their social roles, such as the role of a general-purpose assistant. Furthermore, these standards should be negotiated and agreed upon by all relevant stakeholders. On this alternative conception of alignment, a multiplicity of AI systems will be able to serve diverse ends, aligned with normative standards that promote mutual benefit and limit harm despite our plural and divergent values.

How to Measure Value Alignment in AI

How can we make sure that AI systems align with human values and norms? An important step towards reaching this goal is to develop a method for measuring value alignment in AI. Unless we can measure value alignment, we cannot adjudicate whether one AI is better aligned with human morality than another. The aim of this paper is to develop two quantitative measures of value alignment that estimate how well an AI system aligns with human values or norms. The theoretical basis of the measures we propose is the theory of conceptual spaces (G?rdenfors 1990, 2000, 2014, Douven and G?rdenfors 2020, and Str?ssner 2022). The key idea is to represent values and norms as geometric regions in multidimensional similarity spaces (Peterson 2017 and Verheyen & Peterson 2021). Using conceptual spaces for measuring value alignment has several advantages over alternative measures based on expected utility losses, because this does not require researchers to explicitly assign utilities to moral “losses” ex ante. As proof of concept, we apply our measures to three examples: ChatGPT-3, a medical AI classifier developed by Bajer et al., and finally to COMPAS, a controversial AI tool assisting judges in making bail and sentencing decisions. One of our findings is that ChatGPT-3 is so poorly aligned with human morality that it is pointless to apply our measures to it.

Aligning Artificial Intelligence with Moral Intuitions: an Intuitionist Approach to the Alignment Problem

As artificial intelligence (AI) continues to advance, one key challenge is ensuring that AI aligns with certain values. However, in the current diverse and democratic society, reaching a normative consensus is complex. This paper delves into the methodological aspect of how AI ethicists can effectively determine which values AI should uphold. After reviewing the most influential methodologies, we detail an intuitionist research agenda that offers guidelines for aligning AI applications with a limited set of reliable moral intuitions, each underlying a refined cooperative view of AI. We discuss appropriate epistemic tools for collecting, filtering, and justifying moral intuitions with the aim of reducing cognitive and social biases. The proposed methodology facilitates a large collective participation in AI alignment, while ensuring the reliability of the considered moral judgments.

Events and Opportunities

You may find the following events and opportunities of interest. Note that inclusion in this list does not constitute an endorsement or a recommendation.

You may find the following events and opportunities of interest. Note that inclusion in this list does not constitute an endorsement or a recommendation.

Events:

• LSE Event on Behavioural Science and AI (click for video recording HERE)
• Generative AI at Gartner Data & Analytics Summit 2024 (video recording available)
• Conference on Behavioural Science in the Age of AI and Social Media 2024 (the event was in May 2024, but you can access limited materials online)
• Behavioural Analysis 2024 (the event was in May 2024, but you can access limited materials online)
• World Business Forum, November 13-14, 2024, Sydney, Australia
• MRS Behavioural Science Summit 2024 (call for contributions closed but registration is open for the event in September 2024), London, UK

Vacancies:

JPMorganChase, Chicago, IL, USA

Stockland, Sydney, Australia

HiTechGroup, Sydney, Australia

Resources Group, Sydney, Australia

Progressive Leasing, Remote Location

Your Feedback

What are your thoughts on the Intent Engine? How can AI navigate the complexities of intent (or SHOULD IT) while ensuring ethical outcomes? I’d love to hear your perspective—or simply leave a ?? if this sparked a new idea for you!