When AI Becomes the Threat: Deepfakes and AI Enhanced Phishing

Written by: Celestyn Karuppiah, ASME Secretariat

Originally published at https://asme.org.sg/newsroom/article/292/2024-11-19-when-ai-becomes-the-threat-deepfakes-and-ai-enhanced-phishing

Imagine you are notified that a large sum of money was wired out of your company’s account. Your staff insists the transaction was authorised by you over a video call. But here is the problem – you never made that call.? This exact scenario happened in Hong Kong earlier this year,? demonstrating that deepfakes have moved beyond the realm of imagination into a new reality of risk for businesses.

Artificial intelligence (AI) has transformed our productivity and efficiency at work. Unfortunately, it has also empowered cybercriminals, allowing them to craft more convincing deceptions that are harder than ever to detect. The line between real and fake is blurring, creating a trust crisis for businesses as well as individuals. Here is what SMEs need to know on how to stay vigilant.

When Fake Becomes Real

Just two years ago, spotting a scam was relatively straightforward. Suspicious phone calls from numbers starting with +65 or emails from unfamiliar domains were red flags. An email from “your boss” with a questionable domain like [email protected] was easy to spot, along with old-school tricks such as replacing ‘o’ with zero, or the unfamiliar domain ‘.xyz.’ But what happens when the ‘request’ does not come via an email but through a video call? Imagine a video conference call where your colleagues look, sound, and act like themselves. Except they are not real. Could you distinguish between AI-generated people and real humans? Welcome to the unsettling world of deepfakes.

?The Growing Threat of Deepfakes

Advancements in AI and machine-learning have enabled scammers to manipulate video, audio, and images to create hyper-realistic fake content. In Jun 2024, a deepfake videos of Senior Minister, Lee Hsien Loon circulated widely, amassing over 100,000 views. The video convincingly mimicked his voice, appearance, and mannerisms. Although political figures and celebrities have been the primary targets, businesses are increasingly becoming victims as well. Imagine a deepfake impersonating your company’s CFO, requesting sensitive information or financial transactions. The potential for damage is enormous.

AI-Enhanced Phishing and Deepfake Social Engineering

The Cyber Security Agency of Singapore (CSA) reported that in 2023, around 13 per cent of phishing emails included AI-generated content, making them more difficult to identify. These emails have improved grammar, logical reasoning, and a polished, professional tone. A significant number of phishing attempts also used credible domains, like “.com,” and over half of the phishing URLs employed the more secure “HTTPS protocol,” further enhancing their believability. With this level of sophistication, it is harder than ever for employees to detect scams. A healthy level of skepticism is essential to guard against Deepfake Social Engineering or Phishing.

Steps to Keep You Safe

To reduce the risk of falling victim to deepfakes and sophisticated phishing attacks, businesses should consider the following steps:

1. Adopt AI-Powered Cybersecurity Solutions Implement AI-driven tools that detect and respond to threats in real time. These solutions automate threat detection, reducing the time between identifying and mitigating attacks.
2. Enhance Employee Awareness Invest in AI-powered security training programs that simulate phishing attacks and personalize learning based on employee roles. Regular and targeted training will empower staff to better recognise and respond to AI-enhanced threats.
3. Enforce Multi-Factor Authentication (MFA) Strengthen access controls with MFA across critical systems. This adds an extra layer of protection, making it harder for attackers to gain unauthorized access, even with compromised credentials.
4. Verify Sensitive Communications As deepfakes grow more convincing, implement multi-channel verification for high-stakes communications, such as financial transactions or executive orders, to prevent manipulation through AI-generated content.
5. Adopt a Zero Trust Model A Zero Trust approach assumes all network requests are potentially malicious. By limiting access and verifying users continuously, businesses can reduce the impact of compromised accounts or insider threats.

Staying One Step Ahead The growing threat of deepfakes and AI-enhanced phishing is a reality that SMEs cannot afford to ignore. As these scams become increasingly sophisticated, it is crucial to stay vigilant, educate your team, and invest in robust cybersecurity measures. Do not wait to become the next target—start today by implementing multi-factor authentication, enhancing cyber awareness, and using AI-based detection tools. In a digital age where trust can be so easily manipulated, make cybersecurity a top priority. Protect your business, protect your people, and stay one step ahead of emerging threats.

This bulletin is brought to you by Evvo Labs Pte Ltd (www.evvolabs.com), a CISO-as-a-Service Consultant onboarded by the Cyber Security Agency of Singapore (CSA). For personalized cybersecurity guidance and services, please reach out to Evvo Labs at [email protected].

?