This startup’s hyperrealistic deepfakes will soon have full bodies

Startup Synthesia’s AI-generated avatars are getting an update to make them even more realistic: They’ll soon have bodies that can move, and hands that gesticulate. The new full-body avatars will be able to do things like sing and brandish a microphone while dancing, or move from behind a desk and walk across a room. In this edition of What’s Next in Tech, find out how these AI avatars are about to become even more realistic.

Deepfakes are proliferating online thanks to advances in generative AI. There’s a lot of potential for misuse, but there are a growing number of reasons why you may want a deepfake made of yourself or a loved one, too. Join MIT Technology Review reporters and editors for a fascinating discussion on LinkedIn Live about the rise of deepfakes including the risks they pose and some interesting positive uses. Register for free today.

With bodies that move and hands that wave, deepfakes are about to get a whole lot more realistic.

Synthesia launched its first version of hyperrealistic AI avatars, also known as deepfakes, in April. These avatars use large language models to match expressions and tone of voice to the sentiment of spoken text. Diffusion models, as used in image- and video-generating AI systems, create the avatar’s look. However, the avatars in this first generation appear only from the torso up, which can detract from the otherwise impressive realism.

To create the full-body avatars, Synthesia is building an even bigger AI model. Users will have to go into a studio to record their body movements.

But before these full-body avatars become available, the company is launching another version of AI avatars that have hands and can be filmed from multiple angles. The coming update also makes it far easier to create your own personalized avatar.? MIT Technology Review’s AI reporter, Melissa Heikkil?, recorded the material needed in just 10 minutes in the Synthesia office, using a digital camera, a lapel mike, and a laptop. But an even more basic setup, such as a laptop camera, would do.

So why is this update such a big deal? Well, hands are tricky for AI to do well—even more so than faces, Vittorio Ferrari, Synthesia’s director of science, says. That’s because our mouths move in relatively small and predictable ways while we talk, making it possible to sync the deepfake version up with speech, but we move our hands in lots of different ways. On the flip side, while faces require close attention to detail because we tend to focus on them, hands can be less precise.

Even if they’re imperfect, AI-generated hands and bodies add a lot to the illusion of realism, which poses serious risks at a time when deepfakes and online misinformation are proliferating.?

Read the story to learn more about how these hyperrealistic deepfakes are made and watch the AI avatar Synthesia created of our AI reporter.

There’s never been a better time to add MIT Technology Review to your summer reading list. For a limited time, subscribe and save 25% on unlimited tech news, analysis and insights, plus get a FREE print copy of our 10 Breakthrough Technologies issue.

Get ahead with these related stories:

1. An AI startup made a hyperrealistic deepfake of me that’s so good it’s scary — Synthesia's new technology is impressive but raises big questions about a world where we increasingly can’t tell what’s real.?
2. Deepfakes of your dead loved ones are a booming Chinese business — People are seeking help from AI-generated avatars to process their grief after a family member passes away.
3. Why does AI hallucinate? —The tendency to make things up is holding chatbots back. But that’s just what they do.

Image: Courtesy of Synthesia