New study attempts to align AI with crowdsourced human values
Researchers from the Meaning Alignment Institute have proposed a new approach, Moral Graph Elicitation (MGE), to align AI systems with human values.
Definition: Moral graph elicitation is a method used in moral psychology and ethics research to understand individuals' moral reasoning and judgments.
Graphical Representation: It involves the creation of graphical representations, typically in the form of graphs or diagrams, to map out how people perceive moral issues and the relationships between various moral concepts.
Data Collection: Researchers use various techniques such as surveys, interviews, or experimental tasks to elicit individuals' moral judgments and reasoning.
Mapping Moral Concepts: Participants are asked to evaluate moral scenarios or make moral judgments about specific actions. Their responses are then mapped onto a graph, where nodes represent different moral concepts (e.g., harm, fairness, loyalty) and edges represent the relationships between these concepts.
Analysis: The resulting moral graph can be analyzed to uncover patterns in moral reasoning, identify common moral principles, and understand how different moral concepts interact and influence each other.
Applications: Moral graph elicitation can be used in various fields, including psychology, philosophy, sociology, and behavioral economics, to study moral decision-making, moral development, cultural differences in morality, and the impact of situational factors on moral judgments.
Limitations: While useful, moral graph elicitation has its limitations, such as the potential for oversimplification of complex moral reasoning, reliance on self-report data, and difficulty in capturing the dynamic nature of moral decision-making.
Future Directions: Researchers continue to refine and develop methods for moral graph elicitation, exploring new ways to represent and analyze moral reasoning, integrating computational approaches, and addressing methodological challenges to enhance its validity and reliability.
As AI becomes more advanced and integrated into our daily lives, ensuring it serves and represents everyone fairly is paramount.?
This?study?further argues that aligning AI with users’ goals is insufficient to ensure positive outcomes.?
The researchers argue that aligning AI systems solely with operator intent is insufficient for achieving good AI outcomes.
They state, “AI systems will be deployed in contexts where blind adherence to operator intent can cause harm as a byproduct. This can be seen most clearly in environments with competitive dynamics, like political campaigns or managing financial assets.”
To address this issue, the researchers propose aligning AI with a deeper understanding of human values.
The MGE method has two key components: value cards and the moral graph. These form an alignment target for training machine learning models.
Values cards?capture what is important to a person in a specific situation. They consist of “constitutive attentional policies” (CAPs), which are the things that a person pays attention to when making a meaningful choice. For instance, when advising a friend, one might focus on understanding their emotions, suggesting helpful resources, or considering the potential outcomes of different choices.
The moral graph?visually represents the relationships between value cards, indicating which values are more insightful or applicable in a given context. To construct the moral graph, participants compare different value cards, discerning which ones they believe offer wiser guidance for a specific situation. This harnesses the collective wisdom of the participants to identify the strongest and most widely recognized values for each context.
To test the MGE method, the researchers conducted a study with 500 Americans who used the process to explore three controversial topics: abortion, parenting, and the weapons used in the January 6th Capitol riot.
The results were promising, with 89.1% of participants feeling well-represented by the process and 89% thinking the final moral graph was fair, even if their value wasn’t voted as the wisest.
The study also outlines six criteria that an alignment target must possess to shape model behavior following human values: it should be fine-grained, generalizable, scalable, robust, legitimate, and auditable. The researchers argue that the moral graph produced by MGE performs well on these criteria.
This study proposes a similar approach to Anthropic’s?Collective Constitiutal AI,?which also crowdsources values for AI alignment.
However, study author Joe Edelman said on X, “Our approach, MGE, outperforms alternatives like CCAI by @anthropic on legitimacy in a case study, and offers robustness against ideological rhetoric. 89% even agree the winning values were fair, even if their own value didn’t win!”