The Achilles' Heel of AI: Addressing Security Vulnerabilities in RAG Architectures

A16Z?has a massive AI portfolio and was among the first to come up with AI reference architectures which are widely used. In a security review by trailofbits of an application developed using this architecture, there were many security vulnerabilities identified.

Here's a non-tech, jargon free summary interspersed with some of my own thoughts:

?If you are experimenting with AI in your companies, here are 3 security vulnerabilities you need to plug.

• Data poisoning: If you are using public data sources for your Vector DB, then an attacker can use 'sock puppet accounts' to create a false discussion thread on your community forum or on Reddit. This can then be used to poison the data being used by your LLM for recommendations. This poisoned data could be false product issues or malicious technical solutions that create more security vulnerabilities etc. Imagine asking a bot trained on this poisoned data to help you troubleshoot your wifi router or your phone.?
• Retrieval of non public documents through GraphQL injection: Simplifying the tech jargon, there are bugs in the specific vector database used in this example which allowed a malicious actor to query and access non public documents. This has been reported as a bug to the project. However, it's worth checking for GraphQL injection vulnerabilities in the specific Vector DB that you are using.?
• Denial of service through prompt injection: Just like an attacker can drown a customer facing application in Millions of queries to bring down the service (a DDOS attack), your LLM powered bot could also be tricked into going in a tailspin. This could happen by either too many requests to the server tripping a global resource limit, or by using complicated prompts designed for long & complex querying, slowing down the system for everybody.

?To avoid these security issues, here's what you need to do:?

1. Do a regular audit of the vector database. Whenever the source information is updated, the vector database should also be updated to remove any inaccurate or stale content. This means if any malicious source documents or web pages are deleted, the vector database also needs to be updated. This will protect you against data poisoning attacks.
2. The data pipelines for your RAG system need to be periodically and thoroughly tested using real world data for simulated attacks. The data should be manually reviewed for accuracy & to detect any potential injections.
3. Have separate Vector databases for your public & non-public bots. This will help you avoid potential GraphQL injection attacks.
4. Put query throttling and DDOS protections in place for your LLM based bots. This will help avoid prompt injections attacks on your AI service.

?

In my opinion, for large enterprises, additional RAG best practices would be to use Vector DBs embedded within your enterprise database to avoid data injection issues. If you need to ingest publicly available data, use a separate Vector DB & keep it separate from your enterprise database. Select a Vector DB built for enterprises which will minimize the GraphQL injection threats. Oracle 23ai Database with an embedded Vector database is a good option for enterprises to look at.

????????????????????: ?????? ?????????? ?????????????????? ???????? ?????? ???? ?????? ?????? ???? ?????? ?????????????????????? ?????????????? ?????? ?????????? ???? ???? ????????????????????????.