Democratizing Digital Deception

Will bad actors use digital duplicates of our dead loved ones against us? (Issue #119)

Before we get to today's main topic, some miscellaneous goodies and things worth your attention…

• AI Book Recommendation:?I can't say enough good things about?Co-Intelligence: Living and Working with AI?by Ethan Mollick, a professor at Wharton. Most AI books get mired in jargon and see the?future?transformative power of AI, but Mollick talks about the transformation that has already happened, even if AI never gets any better (which, as he says over and over, it will).
• Two moving articles on dementia?and the impact it has on individuals and their families. First,?this New York Times piece?($) by Katie Engelhart (about a battle within one family) just won a Pulitzer, and don't miss?the episode of The Dailydevoted to the story. Next,?this Wired piece?($) piece by Kat McGowan explores how companion robots might make life easier and happier for people living with dementia.
• Who stole my voice??Don't miss?this New York Times piece?($) about two voice actors alleging that an AI company cloned their voices without permission. Along similar lines,?this Atlantic piece?($) profiles ElevenLabs, a technology company that lets you clone your own voice for $22.
• Jess Piper's?The View from Rural Missouri?newsletter is a recent favorite. Her recent piece about?anti-abortion extremists in the Missouri government?is a scary must read.
• David Sedaris on Seth Meyers?talking about his new children's book?is a laugh riot.
• The Final Frontier returns:?the original model of the U.S.S. Enterprise was?found in an old storage closet?($) after 45 years. Now the question is who owns it?

Practical Matters:

• Sponsor this newsletter!?Let other Dispatch readers know what your business does and why they should work with you. (Reach out?here?or just hit reply.)
• Hire me to speak at your event!?Get a sample of what I'm like onstage?here.
• Please follow me?on?Bluesky,?Instagram,?LinkedIn,?Post?and?Threads?(but not X) for between-issue insights and updates.

On to our top story...

Democratizing Digital Deception

Last time, I shared a microfiction (1,000 words or less), a short science fiction story called "Hacking the Dead" about Trix, a corporate spy who influences the digital duplicate of an equity analyst's beloved dead mother in order to change his mind about Trix's company without him realizing it.

This time, I'll explore how realistic the story is or isn't. You don't have to read “Hacking the Dead” (although it ain’t bad) to understand this week's piece, but fair warning:?Thar Be Spoilers Ahead!

Let's dig in.

The idea behind "Hacking the Dead" is similar to the 2010 movie?Inception, except instead of the covert persuasion happening in dreams it happens in the waking world with dream-like digital ghosts of departed family members.

Like?Inception, in "Hacking the Dead" a small team of people (Trix and her unseen co-workers) create a false reality. This is the kind of thing that used to require whole government agencies dedicated to fake narratives and propaganda, but now technology has democratized it. (See also?this earlier piece?about the movie?Wag the Dog?and deep fakes.)

Digital twins of the dead:?The part of the story most obviously different from our world in 2024 is Alice, the digital twin of the analyst's late mother. Outside of Harry Potter movies or the Haunted Mansion ride at Disneyland, we don't see a lot of moving portraits, let alone pictures that chat with us as if they were flesh and blood people. Oliver (the analyst) chats with Alice whenever he is alone—at work, at home, wherever he has an internet connection. He also visits his mother's grave, where her actual remains lie buried, but when he's there he still talks with her digital twin: her headstone is a giant monitor.

How realistic is this??Very. Digital duplicates of dead people have been around for a while.?Replika, a company that lets you create an AI confidant, started when the founder, Eugenia Kuyda, lost her best friend after a car hit and killed him. She used all their text messages to create a chatbot version of her friend. (There's an alarming 10 minute video about Replika from Quartz?here.) Replika is far from the only such company;?this article?from?Gizmodo?describes several companies that create digital twins by absorbing massive amounts of data about people before or after they die.

In today's digital world, gathering data is relatively easy, but that doesn't mean it's impossible to create digital duplicates of people who died before the internet. Famously, AI pioneer Ray Kurzweil is trying to create a chatbot version of his late father Fred, who died in 1970, by digitizing paper records. Kurzweil's daughter Amy lovingly describes this process in her graphic novel?Artificial: a Love Story.

MIT Technology Review's recent article, "Deepfakes of your dead loved ones are a booming Chinese business" ($) describes how a man named Sun Kai has a weekly videochat with his dead mother's digital twin, created by a company called Silicon Intelligence. The article observes:

Avatars of the dead are essentially deepfakes: the technologies used to replicate a living person and a dead person aren’t inherently different. Diffusion models generate a realistic avatar that can move and speak. Large language models can be attached to generate conversations. The more data these models ingest about someone’s life—including photos, videos, audio recordings, and texts—the more closely the result will mimic that person, whether dead or alive.

Deepfakes abound with wildly varying degrees of quality and convincingness.

Maybe you think you could never be fooled by a digital duplicate? If so, then I urge you to go look at some of?the demos from last week's OpenAI?event announcing the arrival of ChatGPT4.o, a new, more powerful, multimodal AI that you can chat with. The AI's speech is more natural than Siri or Alexa, and the flirty personality of the default female persona can be just plain weird and sexist. Desi Lydic made delightful fun of this?on Tuesday's Daily Show, while?on the Hard Fork podcast?Kevin Roose and Casey Newton dug into what makes the AI so convincing and why it's an "engagement hack" for a product that isn't all that useful... yet.

However, key both to the microfiction and to the digital twins of the dead companies in real life is that nobody is trying to fool people that the dead original of the digital twin is still alive. But there are still ethical questions about the use of digital twins of the dead.

In India right now, Vijay Vasanth, an actor running for office,?released an endorsement video?from his father, H. Vasanth Kumar, a well-known politician who died four years ago of COVID, and this is not the only such example. Can Indian voters be sure that the dead father would approve of his son's policies? Not really.

The least realistic?part of "Hacking the Dead" is the precision with which Trix influences the digital duplicate of Alice by bombarding her with positive articles about Trix's employer, a company about to go public that desperately wants Oliver's approval.

Generative AI is prone to hallucinations, which is a nice way of saying that it can tell plausible lies with a confident swagger that makes them seem true. Gen AI doesn't know what is true and what is false, only what words are statistically likely given its training data. So, for example, if an AI's training data includes a lot of romance novels, the Large Language Model will create prose that includes a lot more bodices and a lot more ripping than actually happens out here in the meat world.

The instructions and articles that Trix covertly presents to digital Alice, therefore, might have had a result distant from Trix's goal because of hallucinations.

Why does any of this matter?

For me, the most unnerving part of "Hacking the Dead" is its plausibility—and I wrote it!

The biggest challenge we humans face in the new AI era isn't what will become of our jobs, although that's a biggie, or whether AI will end humanity?Terminator?style. Instead, our challenge is determining what's real, what's mistaken, and what's enemy action.

Thanks for reading. See you next Sunday.