The EU AI Act - "Materially Distorting Human Behaviour"

As artificial intelligence capabilities advance rapidly, the European Union has proposed legislation to ensure these powerful technologies remain safe, trustworthy and respectful of human rights. The draft AI Act reflects admirable aims - promoting beneficial AI innovation while prohibiting applications that could unduly manipulate, deceive or exploit humans.

The legislation takes a risk-based approach, categorising AI systems by their potential threat level. It outright bans the "unacceptable risk" applications seen as inherently manipulative or exploitative, like social scoring systems judging people based on behaviours or demographics (Article 5(1)(c)).

However, translating these ethical principles into precise legal codes reveals profound challenges. For AI systems deemed "high-risk" due to their potential impacts in areas like employment, education, and core public services, the Act sets critical requirements. These include mandates around risk assessment, transparency, algorithmic auditing and human oversight (Chapter 3, Section 2).

But consistently defining, implementing and enforcing such measures across the myriad high-risk AI use cases will be enormously difficult. Ensuring transparent documentation, real-world accuracy, and meaningful human oversight is a substantial compliance burden.

The Act also bans AI systems designed to "materially distort human behaviour" through subliminal or manipulative techniques (Article 5(1)(a)). But defining the line between acceptable persuasion and unacceptable psychological distortion is immensely complex...

Article 5(1)(a) prohibits AI systems that deploy "subliminal techniques" or "manipulative or deceptive techniques" with the objective or effect of "materially distorting the behaviour of a person or a group of persons."

Much of modern marketing is fundamentally about influencing and shaping consumer behaviours through persuasive techniques like targeted advertising, dynamic pricing algorithms, and optimised messaging. These could potentially be viewed as "manipulative techniques" aimed at "distorting behaviours."

Recital 29 acknowledges this tension, stating the prohibition should not cover "legitimate commercial practices." However, where exactly that line is drawn remains unclear, creating legal uncertainty around reasonable AI-driven marketing tactics.

There are also gaps where certain marketing applications like deepfakes, automated disinformation, and social media recommenders may escape explicit prohibition despite potentially distorting behaviours at scale.

So while well-intentioned, the vague prohibitions raise challenging questions for the marketing field and potentially leave some concerning techniques unaddressed. Ultimately, while a vital step, the Act reveals the profound difficulties in translating nuanced human concepts around psychology, autonomy and rights into precise legal regimes...

Potentially Beneficial AI Applications Facing Legal Challenges

1. Health Coaching AI Apps: AI-driven platforms offering personalised advice and encouragement for healthier lifestyle choices might be scrutinised for "distorting" behaviours, even when the aim is to promote well-being and positive habit formation.
2. AI in Education for Behavioural Growth: Educational AI systems designed to adaptively encourage behaviours like teamwork, empathy, and environmental stewardship could be questioned under the law for influencing student behaviors, despite their positive educational outcomes.
3. AI for Personal Development and Mental Health: AI technologies providing cognitive behavioral therapy (CBT) or mindfulness exercises aiming to alter negative thought patterns could face regulatory hurdles, as their direct intention is to "distort" behaviour towards healthier mental states.

Potentially Harmful AI Practices That Might Evade The Regulation

1. Subtly Coercive Marketing AI: AI systems that nudge consumers towards making purchases by exploiting psychological biases or creating a false sense of urgency might not be explicitly covered under the "manipulative techniques" prohibition if they're framed within the bounds of conventional marketing strategies.
2. Recommendation Algorithms Promoting Echo Chambers: Social media algorithms that reinforce existing beliefs by selectively presenting content, potentially leading to polarisation, might not be directly addressed by the Act if they do not cross the threshold of manipulating behaviour in a clearly definable, material way.
3. Deepfake Content for ‘Entertainment’ Purposes: AI-generated deepfakes used in contexts claimed to be for entertainment or satire might skirt around regulations designed to prevent misinformation, despite their potential to mislead or harm public perception.

Guess we'll have to wait and see in court!

Note: The EU Law is 88k words. That's a lot of words. I/we/it spent much of a day pulling it apart, trying to understand it and looking for 'Legitimate Interest' style loopholes in it. There are five significant ones that I/we/it identified. Doing a dive into each. This is the first and most significant for marketeers.

Research: Both Claude and GPT had goes at helping me extract all the relevant laws - both did OK, Claude was ultimately more useful.

Narrative: Jon 50% Claude 50% after much discussion.

Use Cases: Claude 40% GPT 40% Jon 20% - I just wrote the brief and curated

Obsolete.com | Work The Future