Securing LLM Applications: Identifying and Addressing Risks

On Friday, July 19th,? 2024, a significant Microsoft outage took place that affected flight’s, tv and radio stations and hospitals that used online services. This outage impacted people worldwide. Triggered by a single faulty software update provided by the cybersecurity firm, CrowdStrike that was issued on devices running windows operating systems. An estimation of 8.5 million windows devices were affected.

The global IT outage served as a stark reminder of the extensive web of IT interconnections that sustain our digital infrastructure and of the far-reaching consequences when something goes wrong.

Rigorous Testing Always Needed

From a security perspective, there should be steps taken by cyber security teams before introducing new software updates.? Software testing should be done correctly to find vulnerabilities and rectify them quickly. Different types of devices should have been tested, not focusing on specific types only. The release of the update should have been slow with a small number of?users initially. Some companies like Google have slow updates which can minimise damage and therefore problems can be contained quickly.?

What about Gen AI?

The global IT outage also begs the question: if a single software bug can take down airlines, banks, retailers, media outlets and more around the world, are our systems ready for Generative AI?

The deployment of Large Language Models(LLMs) and Small Language Models(SLMs) is becoming increasingly widespread across many industries. Businesses need to acknowledge the security risks associated with these models and take steps to counter these risks effectively. In this blog, we try to distil the essential elements of identifying and addressing security threats related to Gen AI models and best practices for securing LLM and SLM applications.

Identifying and Addressing Gen AI Security Risks

Gen AI applications face significant security risks, including generating offensive content, promoting illegal activities, and reinforcing biases. These issues emerge from unvetted datasets used in training, pushing for the need for robust data integrity and confidentiality measures to prevent misuse.

Data leakage can occur through private information in training documents and user prompts. Mitigation strategies include data encryption, access controls, and user education. Adversarial attacks exploit Gen AI vulnerabilities, requiring regular security audits and adversarial training to counteract them.

Bias and privacy concerns also pose challenges. Addressing inherent biases in datasets and maintaining user privacy through strict data handling practices are essential for fair and secure Generative AI deployments. Some of the risks that are prone to Gen AI applications include:

Prompt Injections

Prompt injections are similar to SQL injections, this involves manipulating Generative AI by lodging inputs that deceive Gen AI therefore performing a function or functions that Gen AI should not be doing. To mitigate the risks of prompt injections, members of our teams should be made aware of it. We should also limit the sensitive data SLM/LLM has access to and function calling should be used as much as possible so the context of Gen AI does not change.

Model of Denied Service

Model of denied service(MDos) occurs when users with malicious intent issue prompts to Gen AI that take up large amounts of system resources. This leads to denial of service requests.? For reducing the risk of MDos we should apply LLM frameworks such as LangChain which limit the number of steps in input evaluation. Practices such as circuit breakers should also be applied.

Sensitive Information Disclosure

Sensitive information disclosure occurs when a Gen AI application exposes sensitive and confidential information back to the user in an unauthorised manner. To mitigate this we should not give Generative AI more data than it needs to do its job. We should also have in place, input and output guard checks for sanitising input from users and output from Gen AI.

Best Practices for Securing Gen AI Applications

At Galvia, we offer consultancy to large and medium-sized enterprises in issues related to the security and testing of AI systems. To secure Gen AI applications, utilise Red Teaming to identify and address vulnerabilities. Red Teaming is a strategy used in cyber security and military training that involves simulating attacks using both automated and manual injection techniques to test the system's defences comprehensively.?

Red Teaming language model applications are a great way to check if vulnerabilities exist that would cause data or sensitive information to be leaked. Employ resources like a prompt library and Giskard LLM scan for detailed analysis. This tool can also be used on Gen AI. Implement robust access controls and data encryption to safeguard against unauthorised access and data breaches. Ensure ongoing protection with continuous monitoring and regular security audits to detect and respond to threats promptly, maintaining a high level of security and adapting to new risks.

Preparing for Giskard SLM Scan

The model should be a standardised interface. Some metadata should be provided e.g name of app, description. Data set of typical queries should be included.

Conclusion

Generative AI offers many benefits across various industries, but it's crucial to acknowledge and mitigate the associated security risks. Red Teaming serves as a powerful tool to uncover vulnerabilities in Gen AI applications, preventing potential data leaks and safeguarding sensitive information. By adhering to best practices like data encryption, access controls, and regular security audits, we can ensure the responsible deployment of SLM and LLMs, fostering trust and maximising their potential for positive impact.

Reach out to us if you want to learn more: https://galvia.ai/book-a-call/

This contribution has been written by Najeeb Rahman , Security Analyst (Intern) at Galvia AI