Tenedos #005: Can Your Computer Know How You Feel? The Answer is Emotion AI

Think about a computer that can tell if you're happy, sad, or annoyed just by hearing your voice or seeing your face: that's what Emotion AI does. It's a new kind of technology that lets machines "read" our emotions. This is changing how businesses talk to us and can even help doctors understand how we're feeling.

What are the implications and potential consequences when our computers, ever-present in our daily lives, evolve to a point where they can discern, understand, and perhaps even anticipate our myriad of emotions?

Emotion AI: A Subset of Artificial Intelligence

How did you feel about the last commercial you watched? Was it funny or confusing? Did it make you want to buy the product? You might not recall, but machines can remember such reactions. Today's advanced AI can spot and understand our emotions.

This is aiding in the creation of more effective advertisements, enhancing healthcare approaches, optimizing customer service experiences, and refining educational content to better engage students.

This topic is not completely new, here you can find an article written in 2019 from MIT Sloan titled "Emotion AI, explained". It delves into the emerging field of Emotion AI, a subset of artificial intelligence that focuses on understanding, simulating, and reacting to human emotions.

Rosalind Picard the first to introduce Emotion AI

The inception of this field can be attributed to the pioneering efforts of Rosalind Picard, who, over two decades ago, embarked on an audacious journey into the intersection of emotion and machines. It was Picard who first introduced the concept of ‘affective computing’ - a branch of artificial intelligence aimed at deciphering and reacting to the emotional states of humans. This groundbreaking idea proposed that machines could be more than just passive receivers of commands; they could be attuned to the nuances of human sentiment. You can find the original paper here.

The Rise of Emotion AI: Making Machines More Human

In our increasingly AI-driven world, there's a growing desire for technology to not only act but also "feel" human. Perhaps we find comfort in the familiar, or our perception of the world is bounded by our human experiences, reminiscent of the Sapir-Whorf hypothesis that suggests our language influences how we view our surroundings.

Humans have always been inclined to reshape the world in their image.

Case in point: Atlas, the robot by Boston Dynamics, which can perform gymnastics. Despite its mechanical nature, Atlas appears somewhat human-like. Contrast this with the company’s headless mechanical dogs, which evoke unease among many. Imagine a talking Atlas. While it's not a reality yet, our interactions with voice-operated chatbots and virtual assistants are ever-increasing.

Infusing AI with empathy could transform customer experiences, fostering deeper trust and stronger bonds.

How Does Emotion AI Work?

Like how AI chatbots use big collections of information, called large language models, to come up with answers, emotion AI also uses a big pile of data. But the type of data is different.

Step 1: Collecting Data

Emotion AI systems gather data from different places. For example:

Voice data: this could come from recorded customer service calls or videos. For example, a customer might sound frustrated when they can't get a refund, or excited in a video about their new gadget.

Facial expressions: this could come from videos of volunteers' faces. For instance, someone might frown when they're confused, or their eyes might light up when they see a surprise birthday cake.

Body data: things like heart rate and body temperature can tell us about how someone is feeling. For example, someone's heart rate might go up when they're nervous about giving a speech, or their palms might get sweaty when they're anxious about an interview.

Different emotion AI systems might use different types of data. For example, a customer service call center wouldn't need video or body data, while a hospital would find it very helpful.

Step 2: Decoding Emotions

How the data is used to understand feelings depends on what kind of data it is:

Text analysis: tools like sentiment analysis or natural language processing can be used to understand written words. These tools can find certain words or patterns that tell us about someone's feelings.

Voice analysis: machine learning can be used to understand different parts of someone's voice, like pitch and volume, to figure out how they're feeling.

Facial expression analysis: computer vision and machine learning can be used to understand facial expressions. This could be basic expressions like happiness or sadness, or smaller, more subtle expressions.

Body analysis: some emotion AI systems can use body data like heart rate and temperature to figure out feelings. This usually needs special sensors and is mostly used in research or healthcare.

How emotion AI works depends on what it's being used for. But most emotion AI systems will use at least one of these methods.

Step 3: Reacting to Feelings

After figuring out the feelings, the AI system needs to react in the right way. This could be telling a customer service agent that their next caller is upset, or it could be choosing the right content for an app. There are a lot of ways this technology could be used, and people are already finding new ways to use it.

Example Applications of Emotion AI

We display more than 7000 different facial expressions every day and we perceive all of them intuitively.

They are the window into our inner selves. In the video below Ms. Pantic describes how she uses artificial intelligence and machine learning techniques to analyse human non-verbal behavior, including facial expressions, body gestures, laughter, social signals, and affective states.

In this video, she shows how these techniques can be used for the good of people – helping autistic children interpret other people’s facial expressions – or for the bad. She imagines AI enhancing human abilities further, allowing for sight, hearing and even communicating to be aided by computers. However, she urges all of us to protect out own behavioral data from corporate misuse.

Emotion AI in Retail: Real-time Empathetic Marketing

In the dynamic world of retail, understanding and connecting with customers has never been more essential. Emotion AI in retail works at the intersection of artificial intelligence and psychology to decode human emotions from subtle cues like facial expressions, voice tone, and even textual content.

As retail strategies evolve, real-time empathetic marketing emerges as a groundbreaking approach. It's not just about pushing products anymore; it's about resonating with a customer's feelings, creating a genuine bond, and offering solutions tailored to their emotional state.

Bringing Emotion AI into the retail sector holds immense potential for enhancing both the customer experience and business operations. As technology continues to evolve, the line between digital convenience and human connection will blur, creating a more empathetic and efficient retail landscape.

Here are some examples of how the technology could be applied:

• Enhanced Customer Insights: Emotion AI provides a deeper understanding of customer moods, preferences, and pain points, allowing retailers to adjust their strategies in real-time.
• Personalized Experiences: by detecting and interpreting emotional cues, retailers can offer tailored product recommendations, ensuring a more relevant shopping experience for each individual.
• Real-time Feedback: instead of waiting for post-purchase reviews, retailers can gauge immediate emotional reactions to products or services and adjust accordingly.
• Empathetic Customer Service: AI-driven chatbots and service interfaces can respond to customers with empathy, addressing issues in a more human-centric manner.
• Optimized Store Layouts: by analyzing emotional responses, retailers can determine the most engaging store layouts and product placements, increasing sales and customer satisfaction.
• Effective Advertising: Emotion AI can evaluate the emotional impact of advertising campaigns, allowing for adjustments and refining to achieve better engagement.
• Improved Product Development: understanding emotional responses to certain products can guide research and development teams in creating items that truly resonate with consumers.
• Reduced Shopping Cart Abandonment: by identifying emotional hurdles in the purchasing process, retailers can address concerns promptly, leading to higher conversion rates.
• Enhancing Virtual Reality (VR) and Augmented Reality (AR) Shopping: Emotion AI can enrich VR and AR experiences by adapting them based on the user's emotional state, creating a more immersive shopping environment.
• Loyalty Program Refinement: by understanding emotional engagement with loyalty programs, retailers can fine-tune offerings to increase participation and customer retention.
• Data-Driven Decision Making: Emotion AI introduces another layer of data analytics, allowing retailers to make decisions that consider both logical and emotional factors of the consumer journey.

Emotion AI: The Future of Digital Health

Emotion AI could help doctors understand patients better through video calls or even spot when someone's mood changes. Emotion AI can lead to better and more personalized care by understanding patients in ways we've never imagined before.

Here are some use cases of how the technology could be applied:

• Telemedicine: enhance virtual doctor consultations. For example, a doctor could receive alerts if a patient seems particularly stressed or upset during a call.
• Mental Health Apps: apps like "MoodTracker" could use Emotion AI to monitor facial expressions during diary entries, prompting users with coping techniques during moments of distress.
• Medication Reminders: a smart pill dispenser might delay a reminder if it detects the user is feeling particularly overwhelmed, reminding them later when they're more receptive.
• Virtual Health Assistants: chatbots, like "HealthBuddy," can offer advice tailored to a user's current emotional state, providing comforting responses during times of anxiety.
• Wearable Devices: fitness bands could detect stress levels and suggest immediate relaxation exercises or meditation breaks.
• Patient Feedback: a feedback kiosk in a hospital could use facial recognition to sense if visitors had a positive or negative experience, helping administrators pinpoint areas for improvement.
• Rehabilitation Programs: programs like "RehabMood" can adjust exercises based on a patient's emotional readiness, ensuring maximum engagement and recovery.
• Health Education: E-learning platforms might adjust the pace of content based on the learner's emotional state, ensuring they're neither bored nor overwhelmed.
• Elderly Care: devices like "SeniorWatch" can track the emotional health of elderly individuals, alerting caregivers if they detect prolonged sadness or emotional isolation.
• Children's Health: an app like "KidCare" might use animations that react to a child's emotions, making activities like tooth brushing more fun when the child seems reluctant.

Incorporating Emotion AI into the digital health tools can create an experience that's not just efficient but also deeply empathetic and responsive to our human needs.

Ethical Considerations

As Emotion AI becomes increasingly used in various sectors - from customer service to mental health support and beyond - it’s crucial to understand the ethical dimensions surrounding its use. Here, we’ll dive into the main ethical considerations of Emotion AI.

1. Privacy and Consent

• Data Collection: Emotion AI often requires large datasets of facial expressions, vocal intonations, or even physiological data to function effectively. This raises issues related to the collection and storage of such personal and potentially sensitive data.
• Informed Consent: it’s paramount that individuals are made aware when their emotional data is being collected and for what purpose. Without proper consent, the use of such technologies can be seen as invasive.

2. Bias and Inaccuracy

Emotion AI, like all AI models, is only as good as the data it's trained on.

• Cultural Differences: emotions and their expressions can vary widely across different cultures. Without diverse data, there’s a risk that the system could misinterpret or inaccurately gauge a user's emotions.
• Individual Variation: even within a cultural group, emotional expression can differ from person to person. Assuming a one-size-fits-all can lead to misinterpretations.

3. Emotional Manipulation

• Targeted Advertising: with the knowledge of a user’s emotional state, companies could tailor advertisements to capitalize on those emotions. For instance, showing comfort food ads to someone detected as sad.
• Decision Alteration: Emotion AI could be used to adjust the environment based on detected emotions. This could mean changing music, lighting, or even information presented to a user to guide them towards a desired action or response.

4. Depersonalization of Human Interaction

• Replacing Human Touch: while AI can recognize or even simulate emotions, it lacks genuine empathy. Over-relying on Emotion AI, especially in sectors like mental health, can deprive individuals of genuine human connection and understanding.
• Authenticity: there’s a risk of blurring the lines between genuine human emotions and machine-simulated emotions, potentially leading to reduced trust in interpersonal interactions.

5. Security Concerns

• Data Breaches: emotion data, if stored, becomes another target for cyberattacks. A breach could lead to the unauthorized release of deeply personal and sensitive information.
• Misuse: beyond breaches, there’s a potential for misuse of this technology in surveillance or control, particularly by oppressive regimes or entities.

6. The Rights of the AI

• Sentience and Rights: If we reach a point where AI can genuinely experience emotions (as opposed to just simulating them), this opens a Pandora’s box of ethical concerns. Do they have rights? Can they suffer?

Conclusions

The development and application of Emotion AI offer incredible potential benefits, from enhanced user experiences to new therapeutic applications. However, like all tools, its ethical use depends on the intentions and precautions of those wielding it. As we advance in this field, a thoughtful and proactive approach is needed to address these ethical considerations, ensuring that Emotion AI serves humanity without inadvertently harming or manipulating it.

As Emotion AI becomes a bigger part of our lives, it opens up a floodgate of questions.

How much can we trust machines with our emotions? Are our feelings just another set of data to be analyzed and used? If a machine can "understand" our emotions, what separates us from them? Will they ever truly grasp the depth of human experiences? How will society change if our emotions are constantly monitored and evaluated? Can we maintain genuine human connections when machines are always watching and interpreting? And if there's an error in the AI's judgment, what consequences might we face?