The Good, The Bad, and The Manipulated - Adversarial AI

Sunny London today, finally stepping out of the cold spell. The warmth on my face makes me think how often do we take for granted the environment around us, assuming things will function as expected? The same goes for AI. We marvel at its capabilities but often ignore the security blind spots lurking beneath.

Adversarial Machine Learning (ML) Security isn’t a future problem—it’s happening now, and most organisations are completely unprepared.

The Double-Edged Sword

We’ve all seen the rise of AI in cybersecurity automating threat detection, enhancing fraud prevention, and even predicting attacks before they happen. But what happens when attackers flip the script? What happens when the very models designed to protect us become the weak link?

Adversarial ML attacks exploit vulnerabilities in AI systems by subtly manipulating inputs to deceive models. The more scary part It doesn’t take nation-state capabilities to pull off an effective attack. A slight tweak in pixel values can make a facial recognition system misidentify a person. A few carefully crafted words can make an AI-powered chatbot spill confidential data. And in cybersecurity? A manipulated AI model could ignore actual threats while flagging harmless activity.

How Adversarial Attacks Work

Attackers manipulate AI models in different ways:

Evasion Attacks – Altering input data to mislead a trained model. Think of modifying a malware signature slightly so it bypasses AI-powered antivirus.

Poisoning Attacks – Injecting malicious data into training datasets, effectively ‘teaching’ the model to make wrong decisions over time.

Inference Attacks – Reverse-engineering models to extract sensitive information, like personal data or proprietary training data.

Trojan Attacks – Implanting hidden backdoors in models that can be triggered under specific conditions.

Why This Matters Right Now

AI is rapidly being integrated into pretty much everything from financial fraud detection to autonomous driving. But there’s a dangerous assumption that AI models are inherently secure. They’re not. The more we rely on AI to make security decisions, the more we must acknowledge that these models themselves need to be secured.

Most organisations today have zero visibility into how resilient their AI systems are against adversarial threats. The few that do are typically playing catch-up. The security industry needs to stop thinking of AI as just another tool and start treating it as an attack surface that requires proactive defence strategies.

The Rise of GPT Models and DeepSeek

With the rise of AI models like GPTs, DeepSeek, and other large language models, we are entering an era where AI is no longer just an assistant but a decision-maker (Hopefully assisted). These models are becoming deeply integrated into business processes, customer interactions, and even automated cybersecurity decision-making. But what happens when an AI model used for customer support or fraud detection starts providing false positives or worse case leaking confidential information due to prompt injection attacks?

Recent discussions in the AI research community suggest that automating adversarial testing of these models is becoming critical. If attackers can manipulate GPTs or DeepSeek like models, we could see an entirely new type of cybersecurity threat one where AI-driven social engineering, misinformation campaigns, or even automated hacking scripts become mainstream.

Ransomware as a Service and Smart Contracts

Another worrying trend is the evolution of Ransomware as a Service (RaaS) into fully automated smart contract-driven ecosystems. Attackers are now leveraging blockchain smart contracts to execute autonomous ransomware campaigns, ensuring payments, key exchanges, and even negotiation processes happen without any human intervention. This level of automation completely removes the need for human cybercriminals in the chain, NO MORE Negotiation, even the creator themselves cannot generate decryption keys until the contract is complete. making it harder for law enforcement to intervene.

With smart contract-based malware, payments are only released when conditions are met, adding an extra layer of complexity to traditional ransomware mitigation efforts. This deserves a much larger discussion because traditional cybersecurity solutions are not designed to deal with autonomous malware controlled by decentralised contracts.

What Can We Do?

Robust Model Testing – Security teams must stress-test AI models, much like how we pen-test applications and infrastructure.

Adversarial Training – Training AI models with adversarial examples to make them more resilient against manipulation.

Explainability & Transparency – If an AI decision can't be explained, it can’t be trusted. We need interpretable AI models that provide reasoning behind their outputs.

AI-Specific Red Teaming – Security teams should simulate real-world adversarial attacks to assess vulnerabilities before attackers do.

Automated Blockchain Forensics – As RaaS shifts towards smart contracts, security teams need automated forensic tools to trace and disrupt these operations before they spread.

Food for thoughts

AI will shape the future of cybersecurity, but only if we secure it first. We’ve spent decades refining traditional security defences, yet AI remains largely unexplored from a security standpoint. It’s time to change that. The attackers are already experimenting are we?

This is a discussion that needs to happen now. How are you preparing for adversarial ML threats? Have you encountered AI security challenges in real-world deployments?