My Recent AI Translation Experiments

While I would presume that we're all aware of the adversarial uses of deep fake technology and AI to convince people to do things that are going to end badly like wire transfer $10MM to the Cayman Islands because you really thought your CFO requested it or pay to have a loved one released from a purported overseas incarceration that may or may not include a death threat, I thought it would be nice to focus on one of the positive uses of that same technology: translation.

This short piece of writing about recent advances in AI technology was not, I will point out, written with the help of any LLM (Large Language Model). It's pure human knowledge and original wordcraft coming to you unmediated, moderated, curated or crowdsourced. ;-)

Embracing Technology

Given the fact that I'm an adjunct professor teaching cybersecurity courses at 美国纽约大学 for the last few years and that I've spoken at various cybersecurity conferences such as Black Hat, SANS, Gartner and GovWare it should come as no surprise that the internet does not want for samples of my speaking voice. So it is without trepidation that I embraced this experiment at seeing just how convincing a recent entrant to the scene really is that has a video translation service called HeyGen after someone else's experiment with the tool showed up on my social media feeds. Yes, it looked impressive with regard to lip sync and the voice emulation was quite natural and smooth. The translations themselves were spot on as far as I could tell (in addition to English I speak conversational Spanish and Dutch).

If you don't know of at least three ways of abusing a tool, then you don't know how to use it.

My experiment does not push the envelop of AI translation capabilities in any way, shape or form. But the spirit behind the quote above is what typifies the motivation of many "hackers" in the sense of the word that means experimenters and those with a curiosity about how things work. The word "hacker" should never be used as synonymous with digital criminals. In my view, a hacker is someone who turns a doorknob to see if it is locked, not to steal or destroy that which lies on the other side of the door. Hacking, in this broad sense, is merely trying to see if you can get a system or a tool to behave in a manner that is not what the creator of that thing intended.

The Demo Clip

Here is the original video clip that I recorded for this experiment. Each of the subsequent video clips is a translation that was cooked up by some impressive AI video deep fake technology. "Deep" in the sense that the quality of emulation of the original clip is non-trivial. There is a good effort to make my lips move in such a way as to match the sounds of the words being spoken. "Fake" in the sense that I did not speak these words and the artifact produced by the translation tools is not "original" or "authentic." The message of my clip was improvised and an entirely spontaneous bit of explication of what I was doing and why. It took several days for me to get these clips translated. Admittedly that’s more a factor of the popularity of the platform at the moment (queue depth of 20k - 170k videos at any given moment) and my reluctance to pay for a subscription to skip ahead in the line.

English

Spanish

French

German

Japanese

Mandarin

Hindi

Dutch

If you'd like to give this tool a try, you might as well use my invite link so that I can get a couple of free credits to continue experimenting with it, yes? :-)

https://app.heygen.com/guest/templates?cid=5d02026e

My next experiment might be to record something a bit lyrical with changes in pitch and maybe also have a metronome running in the background to keep track of the temp and see if it pays attention to that or not.