Enhanced detection of adversarial attacks on AI ML (files, audio, video and signals)

As part of Entrepreneurship MBA project at The University of Queensland, my team is working with CSIRO's Data61. Our task is to find opportunities to commercialese the invention to improve the accuracy of detection of adversarial attacks. The algorithm is capable of running on classical computers using high performance computing (HPC).        

Attack Scenarios

As AI increasingly makes real-time decisions with significant responsibility and trust, the potential for adversarial attacks grows. Consider these scenarios:

1. A driverless car misinterprets a manipulated stop sign as a 120 km/h sign.
2. A machine-readable invoice appears correct to humans but is altered to redirect payments.
3. A CEO’s Twitter/X account, grown to hundreds of thousands of followers, is hacked to release a deep fake video, causing stock prices to plummet.
4. Facial recognition on mobile devices could be tricked into verifying the wrong person for bank accounts, loans, or property sales.
5. Healthcare, Financial, Mining companies and other medium to large enterprises rely on a web of SDKs and APIs to transport on-prem and cloud data, images, and audio on site, between sites and to and from your customers and suppliers without verifying if this original data has been altered in any way before decisions are made that could cost the business millions.

Processing of video frames in near real-time

Classical computing with high-performance computing (HPC) may be fast enough to detect still images within near real-time however a CCTV video of a person fooling an automated passport control into thinking that they are someone else won't be detected until the person has left the airport. This is where Quantum computing as a service (QcaaS) will truly add value.

Both images clearly look like a STOP sign to the human eye, but with some manipulation using techniques such as Carlini & Wagner's image manipulation machine learning could read it as 120 km/h – a serious risk for motorists resulting on head on collisions

For a web-based demo on how this works see adversarial.js – Intro (kennysong.github.io) by Kenny Song.

Gartner’s research highlights that quantum computing poses significant security risks, including the potential to break current cryptographic algorithms. This could lead to vulnerabilities such as compromised secure communications and identity management. To mitigate these risks, organisations must evaluate their cryptographic practices and prepare for a post-quantum cryptography landscape.

Potential Use Cases

1. Autonomous Vehicles (Self-Driving Cars)

Problem: Autonomous vehicles rely heavily on machine learning models to process video data from cameras and LIDAR systems. Adversarial attacks on these models could involve subtly altering the visual input to trick the car into misinterpreting road signs, lanes, or obstacles.

For example, a small perturbation to a stop sign might cause the AI to misclassify it as a yield sign, leading to dangerous driving decisions.

Solution: A quantum-based adversarial defense algorithm could detect even the smallest perturbations in the video stream, ensuring that the vehicle accurately interprets road signs, pedestrians, and obstacles, reducing the risk of accidents caused by adversarial attacks.

2. Healthcare AI Systems

Problem: Machine learning models are increasingly used in medical imaging systems (e.g., for detecting cancerous tumors in scans). Adversarial attacks could modify medical images in subtle ways, leading to incorrect diagnoses.

For instance, an adversarial attack could manipulate an MRI scan to hide signs of a tumor or make a benign growth appear malignant, leading to misdiagnosis or unnecessary treatments.

Solution: A quantum algorithm could detect these minute perturbations in imaging data, protecting the integrity of medical diagnoses. This would ensure that the AI provides reliable diagnostic assistance, critical in high-stakes medical environments.

3. Voice-Activated Systems (Speech Recognition)

Problem: Voice-activated systems, such as virtual assistants (e.g., Siri, Alexa), call centres, or voice-based authentication systems, are vulnerable to adversarial attacks. An attacker could modify an audio signal in a way that’s imperceptible to the human ear but causes the speech recognition system to misinterpret the command.

For example, an adversary could modify a voice command meant for a banking assistant, tricking the system into transferring money or performing unauthorised actions.

Solution: A quantum adversarial defense could be applied to audio data, detecting subtle adversarial perturbations and ensuring that the system correctly interprets legitimate commands while blocking malicious ones. This would improve the security of voice-based systems, which are increasingly used in sensitive applications.

4. Fraud Detection in Financial Systems

Problem: Financial institutions rely on machine learning models to detect fraudulent transactions. Adversarial attacks could manipulate transaction data in a way that evades detection, allowing fraudulent activities to go unnoticed.

Fraudsters could use adversarial techniques to trick AI models into classifying a high-risk transaction as low-risk, facilitating money laundering or other financial crimes.

Solution: A quantum algorithm could be integrated into fraud detection systems to identify adversarial manipulations in transaction data. By preventing adversarial attacks, the system would better protect against fraud, improving the security of financial transactions and reducing losses.

5. Adversarial Attacks on Facial Recognition Systems

Problem: Facial recognition systems are commonly used for authentication in smartphones, security cameras, and access control systems. Adversarial attacks could involve altering the input (e.g., wearing adversarial patches or glasses) to trick the system into either misidentifying a person or allowing unauthorized access.

For example, an attacker could wear adversarial glasses that make them appear as someone else to a facial recognition system, gaining unauthorized access to secure areas.

Solution: A quantum-based algorithm could be used to detect adversarial perturbations in facial recognition inputs, ensuring that the system recognizes individuals correctly, even in the presence of adversarial modifications.

6. Optical Character Recognition (OCR) Systems

Problem: OCR systems are used to digitize text from scanned documents, images, and PDFs. Adversarial attacks could involve modifying the visual representation of text so that the OCR system misinterprets critical information.

For example, an attacker could modify a scanned legal document in a way that makes the OCR system misread key clauses, potentially leading to contractual disputes or legal fraud.

Solution: A quantum algorithm could be applied to OCR systems to detect adversarial manipulations in the text, ensuring that the digitized version accurately represents the original document. This would be especially valuable in sectors like law, finance, and government, where document security is critical.

7. Cybersecurity and Network Defense

Problem: Adversarial attacks can be used to compromise machine learning-based cybersecurity systems. For example, attackers could trick an intrusion detection system (IDS) or malware detection system into ignoring malicious traffic or flagging benign traffic as suspicious.

Adversarial examples could be crafted to evade detection, leading to data breaches, ransomware attacks, or other serious cybersecurity incidents.

Solution: A quantum-based adversarial detection algorithm could identify adversarial patterns in network traffic or malware signatures, improving the robustness of machine learning-based cybersecurity tools. This could help prevent sophisticated cyberattacks that exploit adversarial vulnerabilities.

8. Robotics and Industrial Automation

Problem: Machine learning is used in robotics and industrial automation systems to enable tasks like object recognition, navigation, and quality control. Adversarial attacks could disrupt these systems by causing robots to misinterpret their environment, leading to operational errors or even safety hazards.

For instance, an adversarially perturbed image could cause a manufacturing robot to misclassify a defective product as acceptable, leading to faulty products being shipped to customers.

Solution: A quantum algorithm could detect adversarial manipulation of sensor data, ensuring that robots and automated systems function correctly, even in adversarial environments. This would enhance the reliability and safety of industrial processes.

9. Military and Defense Applications

Problem: Machine learning models are used in defense systems for tasks like target recognition, surveillance, and autonomous drone operation. Adversarial attacks could be used to mislead these systems, causing them to misidentify targets or make erroneous decisions in combat situations.

An adversary could manipulate video or sensor data to make a defense system misclassify an enemy vehicle as a civilian one, leading to incorrect tactical decisions.

Solution: A quantum-based adversarial defense algorithm could protect military AI systems from adversarial attacks, ensuring that critical defense systems operate reliably and securely, even under attack.

10. AI-Based Content Moderation

Problem: Social media platforms and online services use machine learning models to detect harmful content such as hate speech, misinformation, or explicit material. Adversarial attacks could be used to bypass these content filters, allowing harmful content to be posted or spread undetected.

For example, an adversary could subtly modify an image or text post in a way that evades detection by content moderation algorithms, allowing harmful content to spread.

Solution: A quantum algorithm could enhance the ability of content moderation systems to detect adversarial examples, ensuring that harmful content is flagged and removed even if it has been deliberately crafted to evade detection.

10. Data Centre Scheduled Jobs Optimization

At the core of both adversarial attack detection and job scheduling optimization are complex mathematical problems that can be framed as optimization tasks. Quantum computing excels in solving certain types of optimization problems more efficiently than classical methods. Let’s explore how this might work for data center job scheduling:

a) Quantum Optimization Algorithms

Quantum algorithms, like Quantum Approximate Optimization Algorithm (QAOA) and Quantum Annealing, are designed to solve combinatorial optimization problems, such as:

• Resource Allocation
• Task Scheduling
• Minimising Costs or Energy Usage

These algorithms can evaluate many possible configurations simultaneously (thanks to quantum superposition), making them potentially far more efficient than classical algorithms.

b) Job Scheduling as an Optimisation Problem

In a data center, job scheduling typically involves:

• Allocating computational resources (CPUs, memory, storage) to various jobs.
• Minimising job completion time (makespan) or energy consumption.
• Balancing loads across servers to avoid bottlenecks or underutilisation.

These same techniques could be adapted to optimize the allocation of resources in a data center, by:

• Minimizing energy consumption while maintaining performance.
• Reducing latency by more efficiently scheduling jobs.
• Maximizing throughput by ensuring all resources are utilized effectively.

Conclusion

In various real-world applications, adversarial attacks pose a significant threat to the reliability, security, and accuracy of AI systems. A quantum computing algorithm for preventing adversarial attacks—such as the one described in patent WO2024/086876 A1—could provide a cutting-edge solution to these challenges by offering more effective detection and mitigation of adversarial perturbations than classical methods. This would greatly enhance the robustness of AI systems in domains like autonomous vehicles, healthcare, cybersecurity, finance, content moderation, and more, ensuring safer and more reliable operation in adversarial environments.

I’m reaching out to my LinkedIn network: if this topic resonates with you or if your organisation is facing similar challenges please send me a message or add a time in my diary here: Book Appointment.

References

Data61's Patent WO2024/086876 A1 Patent [PDF file]

Data61 projects and tools [https://research.csiro.au/data61/]

Can quantum computing protect AI from cyber attacks [https://www.csiro.au/en/news/All/Articles/2023/May/quantum-cyberattacks]

Data61’s Year in Review [https://algorithm.data61.csiro.au/data61s-year-in-review/]

CSIRO's Data61 develops 'vaccine' against attacks on machine learning [https://www.zdnet.com/article/csiros-data61-develops-vaccine-against-attacks-on-machine-learning/]/

Agenda | Quantum Computing is on the Rise - Are You Prepared? (gartner.com) By Christian Stephan, Senior Director Analyst for Innovation & Disruption at Gartner