A Cure to Hallucinations and Data Leaks?
Generative AI is transforming businesses - but safely implementing it takes care.
One of the hottest business initiatives is developing rigorous standards for AI use. This has been sparked due to concerns about hallucinations, opaque use of training data, and the risk of internal data leaking out. What if I told you many of these problems have already been solved?
Let's demystify how to mitigate those risks using Retrieval Augmented Generation with today's technology.
Stop the leaks...
First, you can and should control all your data.
Before we get too far, there is a critical myth we should dispel. Using an LLM does not mean everything you type (your prompt) is instantly sucked into Skynet forever. Models are baked goods. When you talk to ChatGPT or any other LLM, your words do not immediately impact their capabilities. They are not learning from you, and the data you have provided is not instantly part of their brain.
Some online LLM services retain prompt data to make improvements and train their future models. This is where the risk of a data leak comes from. We can avoid this problem altogether.
It’s now possible to use models hosted entirely on internal systems, removing the need to send data to a 3rd party. Depending on the scale of your business, you may be able to host an LLM in your existing data center, and you can most definitely host them with your cloud provider. It's not as impossible as you may think! So, step 1 is hosting the model yourself.
Open Source, You Say?
Foundational open-source models have been released. They offer visibility into their work and the data used to train them. Open LLMs also offer controls allowing us to customize and fine-tune them. Community-driven development can also improve security and expand capabilities faster than commercial products. These models are released in several flavors that scale in parameter count.
The availability of models with fewer parameters means they require fewer resources to run. There is a trade-off between capability/knowledge and required resources with fewer parameters, but we often don't need all the power of "GPT-4". A smaller, high-quality LLM will work perfectly fine for many tasks. Models such as Llama 2 13b are competent and can be tweaked to run on existing affordable GPU services. We can leverage quantization to scale down the hardware requirements further or improve performance with small trade-offs in accuracy.
So you can choose an open-source model that fits your standards and implementation budget, then bring it inside, where you don't need to worry about exposing data to another company.
Focus Daniel-san!
Now that we’ve mitigated data leakage and addressed some ethical and compliance obligations, how do we mitigate "hallucinated" content? Constraining your new LLMs’ knowledge sources with significant effect is possible. We need to convert our company data into a format better suited for the AI to understand and leverage. While it's possible to point the LLM at a conventional database, leveraging a vector store can improve performance and accuracy.
领英推荐
NLP tools can extract essential information from approved internal data. Here open-source encoders like Hugging Face's SentenceTransformers are handy. They encode relevant data from documents into the vectors AI understands. Then we can save the encoded data in a special vector database for lightning-fast querying. Now that the AI can speak to our data in its native language it's time to tie it all together.
We can use open-source tools like LangChain to integrate the databases with the AI model, allowing it to provide accurate, constrained, and cited results. We can even develop a solution to hyperlink the original content in its responses. Now we have our own open, transparent, accurate, secure, and firewalled LLM that can operate with our data, safely.
Scaling Up Responsibly
When we have the AI pipeline developed, containerization with Docker enables scalable deployment of specialized, highly accurate agents and seamless integration into other tools where we can deliver:
The solution outlined here shows one way forward but may only work for some use cases. The key is having an open conversation about requirements and potential solutions. With the right architecture, developing responsible and beneficial AI is within reach.
Harnessing Generative AI Responsibly
Vendors have started offering responsible AI features and are working quickly to keep pace with open-source innovations, integrating similar capabilities into their products. The open-source community has enabled control over AI systems' security, accuracy, and auditability. This proves that large language models and other AI technologies don't have to be opaque "black boxes" - with the right architecture, developing responsible and beneficial AI is within reach.
The landscape has changed dramatically in just the past three months and will look completely different by next year. There are new breakthroughs nearly every day across all facets of AI, from scaling and safety to transparency and reliability. The solutions will only grow more capable and trustworthy over time.
By combining industry resources with open-source innovation, we can ensure AI progresses equitably and ethically. There is always more work to be done, but the future is bright for responsibly developing and harnessing AI's immense potential.
Next week I will discuss another way to control hallucinations and compliance through LLM Training and Fine Tuning.?
I'm happy to chat about leveraging AI responsibly or get under the hood of generative AI solutions anytime.