In the memory of deepfakes

Last weekend was the Memorial Day weekend. For those outside of the United States, it is a holiday honoring and mourning the U.S. military personnel and marks the start of the summer season.

Humans have multi-dimensional and occasionally paradoxical reactions to death. While the process of remembrance can often be a source of pain and loss, we just as commonly find inspiration and strength from it.

In 2022, just months before ChatGPT made its first debut, Amazon announced a new feature for Alexa — while it was still in development at the time, it was designed to replicate a family member's voice, even after they've died. The general public's reaction was mixed. Some saw the feature as a touching way to remember loved ones, while others felt it crossed a line and could be potentially harmful. As I remember, most people simply found it "creepy."

More recently, MIT Technology Review reported a new trend in China where people are turning to deepfakes of their deceased loved ones as AI-generated avatars to cope with their grief.

These diffusion models create lifelike avatars capable of movement and speech, and large language models enable these avatars to hold conversations. The more data these models consume—such as photos, videos, audio recordings, and texts—the more accurately they can mimic the individual, whether alive or deceased. This raises serious questions about the potential misuse and ethical implications of such technology.

In the ever-evolving realm of artificial intelligence, it is crucial to tread the fine line between using AI as a tool to amplify human qualities and the potential risk of it replacing human connections entirely. The concept of a machine replicating a person after their death raises profound ethical concerns about consent and the potential for misuse. Furthermore, it can blur the boundary between reality and simulation, potentially impeding the grieving process and causing emotional turmoil.

When we recall a memory, the context in which we do so can subtly, and sometimes significantly, alter it. Memories that we revisit frequently can change quite dramatically over time. They can be imbued with emotions they initially lacked, given new interpretations, or even intertwined with other experiences. This dynamic nature of memory is essential to our emotional growth and personal history. And it is also because the process of honoring a person in your memory can be so dynamic, it should not be so readily supplanted by your interaction with an AI avatar.

Around the time that Amazon launched Alexa's new voice feature, I lost my mother to a long battle with cancer. She was my mentor and a close confidant later in my adult life. However, as a practitioner of Buddhism, she often advised me not to be imprisoned by over sentimentality. Before she passed, she reminded me that understanding and accepting the impermanent nature of life is not about dismissing the pain but recognizing it as a natural part of existence.

AI does not experience pain or death. It is not concerned with or understand human conditions. By striking the right balance, we can still harness the power of AI to create a future where technology enriches human experiences. But it should always be a tool that amplifies our humanity, not a substitute for it.