Magically Multilingual
Joshua Peskay
3CPO (CIO, CISO, CPO) CISSP, CISM - Helping nonprofits leverage technology to do more, do better and be more secure. Also, I collaborate with a potato.
I made a short video this morning.
In the video, you will see me appear to fluently speak six (6) different languages (English, Spanish, Portuguese, French, German and Hindi).
In reality, I only speak one language, English (and barely that).
The entire process took me approximately 30 minutes. I used a service called HeyGen.ai, which specializes in generative video. HeyGen is partnering with 11elevenlabs.io which specializes in generative voice. Together, they have reached a level of effectiveness in this that is, to me, astonishing.
I sent a couple of earlier videos to some colleagues as a joke last week and the majority did not think it was a joke. They thought I had been hiding a secret language fluency! I have played some of these for native speakers of these languages and they have told me that is near flawless - in other words, they would swear I was a near native speaker of their language. I encourage you to watch and skip through to the different languages (especially if you are a fluent speaker of Spanish, Portuguese, French, German or Hindi). Watch the lips closely, I don't think you'll be able to tell, at all, that it's a modified video. It's stunning.
领英推荐
For ANYONE who can benefit from providing video in different languages, you, RIGHT NOW, have the ability to do so and do so at an incredibly high quality and at (I think) quite modest cost (roughly $1 x minute). Currently the only languages available are the six I included plus Polish & Italian, but I am certain many more are coming soon.
Think, for a moment, about where this is going. Zoom, for example, has had translated captions available for over a year now. Imagine when this technology is good enough that you can have a real-time video meeting with someone with whom you do not share a language, and yet, you can communicate flawlessly with one another. We are getting closer and closer to Babel fish territory and I find it wonderfully exciting.
Pop Culture Curator, Interpretive Planner, Interactive Designer, Writer & Editor
1 年Josh, I have questions! Did you use generative voice cloning to create the English portion as well? Did the system do your translations for you, or did you need to provide text in each language? Details please!
Senior Consultant at Exponent Partners
1 年That is bonkers. Impressive, useful, and creepy all at once. I particularly enjoyed “je suis Joshua Pesque.”
Charity IT Consultant & Business Analyst
1 年Guessing it currently requires the speaker to speak slowly and clearly enough, but it's impressive. Can't wait for it to work effectively for live conversations! I was convinced by the 4 out of 6 I could follow. You really nailed the English version, btw! ;)