Anticipating the Mainstream Emergence of AI-Powered Voice Scams

It's Saturday, March 30, 2024, just a day before Easter, and Sarah, a devout follower of a popular "prophet," receives a call from what appears to be the seer himself: Benini Skyseeker (fictional name).

She's overwhelmed and cannot believe her ears and eyes as "Benini" claims to be reaching out to random members of his congregation, whom he had seen in a vision.

The conversation centers around Easter, but before long, "Benini" feigns an emergency and has to attend to another pressing matter.

And just before he leaves, he apologizes and asks Sarah to call him back about an hour later to continue the conversation. Thus began Sarah's anxious one-hour wait.

She tells her brother and a friend about the surprise "video" call, but they don't believe her. I mean, they're all registered congregants, but why would Skyseeker call her from the 112,000 members?

When the day-long hour finally elapsed, Sarah called back, with her brother eavesdropping in the background. They are both greeted by the same deep, resonant, and powerful voice of Benini Skyseeker, whom she has known for at least a decade.

Unfortunately, the prophet can only speak now, no longer on video.

He asks a few personal questions and repeats a message about "detachment" from his Good Friday sermon, encouraging her to donate a special "Easter" offering to a cause announced a week earlier.

The voice reassures her that the donation would bring her and her family immense blessings and good fortune.

Although her brother protests, that he had not seen the prophet and that any voice actor (and even AI) can mimic the prophet, Sarah is completely taken in by the authenticity of the voice and the video she had seen earlier on the first call.

She feigns agreement with her family but still sends the requested "Easter offering" to the given account with hopes of blessings and good fortune.

However, a few days later, hundreds of people report getting the same call (voice and video) from whom they swear sounds and looks like 'the prophet.'

But soon enough, Sarah realizes that she has been scammed after Skyseeker appears on a live TV show to issue a statement, warning his followers about fraudulent calls made in his name and advising them to report any such incidents to the authorities.

Sarah is devastated by the realization that she has fallen prey to an AI-generated voice (and deepfake) scam. She feels violated, exploited, and foolish for trusting a stranger (albeit, a familiar voice).

She contacts law enforcement and her telco company, to report the incident and finds out that many other customers have also been scammed in a similar manner.

Despite being a work of fiction, Sarah's experience serves as an indicator of the imminent danger posed by AI-powered voice and deepfake scams.

The utilization of advanced AI algorithms by malevolent actors with ulterior motives that conflict with the public interest will be the reason behind the proliferation of such scams in the very near future.

On April 3, 2023: Researchers noticed a 135% increase in 'novel social engineering attacks' from January to February 2023; corresponding with the widespread adoption of ChatGPT

So, it can be inferred that as other AI domains reach new levels of sophistication (text-to-voice AI and voice-to-voice AI), we will be presented with a plethora of new opportunities and cybersecurity challenges unlike anything seen before.

Fortunately, telcos can take inspiration from China Mobile's Tiandun, or Sky Shield to prepare for the coming onslaught.

Tiandun is a big data anti-fraud system that utilizes several techniques, such as deep packet inspection, behaviour analysis, and machine learning, to identify and block malicious traffic in real time.

Through the incorporation of machine learning technology and analysis of a large number of fraud cases, Tiandun can recognize fraudulent communication behaviour and differentiate it from normal calls, intercept spam calls and texts, and warn vulnerable users via text messages.

Additionally, when the system detects potential fraud, it will send the relevant phone information to the police, who can quickly contact the user for further action.

Certainly, there are privacy concerns when implementing a system like Tiandun. Yet, it offers a practical solution for combating scams that will be facilitated by AI (text-to-voice and voice-to-voice).

Piloted in Zhejiang Province, Skyshield has reportedly intercepted fraudulent phone calls from international and domestic networks, and accomplished risk assessments for victims during the calling process, with the accuracy rate of potential victim information sent to police stations reaching 80% (according to China Mobile).

What's Next?

As noted in the Applied AI Course for Business Leaders, telecom companies, and social media companies with instant messaging apps must collaborate better than ever before to build 'non-intrusive' AI-based systems that can detect and identify fraud via traffic analysis.

We must not wait for a "Sarah" story to happen before initiating programs like Tiandun, to shield the vulnerable public.

At the moment, telcos are deeply integrating AI into their core systems, especially with the advent of 5G technology. So the possibility of "AI-based" services for fraud detection and customer protection is not a distant reality.

Nevertheless, Telcos must be more proactive by simulating AI-powered voice scams and deepfake scams, starting now. The fabric of digital trust rests on this singular objective.

PS: Some of the models and tools for building an anti-fraud AI system are discussed in the Applied AI Course for Business Leaders.