Three Ways to Incorporate Your Own Data When Working with Generative AI

A quick guide for business people.

Generative AI, via the likes of Open AI's GPT-4, Google's Gemini and Anthropic's Claude, is revolutionising the way businesses interact with and utilise artificial intelligence. These models can generate human-like text, assist with various tasks, and provide intelligent responses to user queries. However, to truly harness the power of generative AI for your business, it will often be the case that you wish to incorporate your own data into the process. In this article, we'll explore three methods to integrate your company's knowledge with generative AI and discuss the pros and cons of each approach.

Method 1: Uploading Documents within the Context of a Question

The simplest way to incorporate your own data when working with generative AI is to upload relevant documents within the context of a question. This approach involves providing the AI with specific excerpts or documents that contain the information needed to answer a given query. By doing so, you can ensure that the AI has access to the most relevant and up-to-date information from your company's knowledge base.

Pros:

• Easy to implement and doesn't require any additional technical setup
• Allows for targeted information sharing based on the specific question
• Provides a straightforward way to control the information the AI uses to generate responses

Cons:

• Limited by the size of the AI's context window, which may not accommodate larger documents or extensive knowledge bases
• Requires manual effort to select and upload relevant documents for each question
• The AI may still generate hallucinations if the provided context is incomplete or if it makes assumptions beyond the given information

Method 2: Connecting a ChatGPT Model to an Internal Directory of Documents

Another approach is to connect a model to an internal directory of documents. This method involves setting up a system where the AI can access and retrieve information from a centralised repository of your company's documents, such as reports, presentations, and knowledge base articles. One might do this via creating a 'GPT' via ChatGPT, or by interacting with a models API.

Pros:

• Enables the AI to access a larger volume of information compared to uploading documents within the context of a question
• Allows for a more automated and scalable way of incorporating your company's knowledge into the AI's responses
• Provides a centralised and organised way to manage and update your company's knowledge base

Cons:

• Requires technical setup and maintenance to connect the AI model to the document directory and ensure smooth information retrieval
• The relevance of the retrieved information may be limited by traditional keyword-based search techniques
• The AI may generate hallucinations if the retrieved documents are not entirely relevant or if there are gaps in the information

Method 3: Using a Vector Database and Retrieval-Augmented Generation (RAG) with Claude or A N Other model.

The most advanced and effective way to incorporate your own data when working with generative AI is to use a vector database and Retrieval-Augmented Generation (RAG). This approach involves converting your company's knowledge into high-dimensional vectors and storing them in a vector database. The RAG technique is then used to retrieve the most relevant information from the vector database and integrate it seamlessly with the AI's generation process.

Pros:

• Offers the most accurate and relevant information retrieval based on semantic similarity
• Enables a tight integration between the retrieved information and the AI's generation process, minimising the risk of hallucinations
• Provides the highest level of customisation and fine-tuning to suit your specific business needs

Cons:

• Requires significant technical expertise and resources to implement and maintain
• Involves additional costs for setting up and running a vector database
• Requires careful data preparation, preprocessing, and vectorisation to ensure optimal performance

Choosing the Right Approach

The best approach for incorporating your own data when working with generative AI depends on your business requirements, available resources, and the desired level of customisation and accuracy. If you prioritise ease of implementation and have a limited amount of data to incorporate, uploading documents within the context of a question may be sufficient. If you have a larger knowledge base and want a more automated solution, connecting a ChatGPT model to an internal directory of documents could be a good choice.

However, if your primary goal is to minimise hallucinations, maximise the relevance of the AI's responses, and have the highest level of customisation, using a vector database and RAG with one of the foundational models (GPT, Claude et al) is the most suitable option. This approach ensures that the AI is always grounded in your company's factual information and provides the most accurate and reliable responses.

Regardless of the approach you choose, it's essential to regularly review and update your company's knowledge base, monitor the AI's performance, and have human oversight to identify and correct any potential inaccuracies or hallucinations.

By incorporating your own data into generative AI, you can unlock the full potential of this technology and create AI-powered solutions that are tailored to your business needs.

A Note on Hallucinations:

When it comes to minimising hallucinations and ensuring accurate information (such as with a customer service chatbot), the choice of approach can have a significant impact.

A) Uploading documents within the context of a question:

Hallucination risk: Moderate to High

Explanation: While providing relevant documents within the context can help ground the responses in factual information, there is still a risk of hallucinations if the model makes assumptions or extrapolates beyond the provided context. The limited size of the context window may also lead to incomplete information, increasing the chances of hallucinations.

B) Connecting a ChatGPT model to an internal directory of documents:

Hallucination risk: Moderate

Explanation: By connecting the model to a directory of documents, you can provide a larger volume of information for the model to draw from. This can help reduce hallucinations compared to the first approach. However, the effectiveness of this approach depends on the quality and relevance of the documents retrieved by the keyword-based search. If the retrieved documents are not entirely relevant or if there are gaps in the information, the model may still generate hallucinations.

C) Using a vector database and RAG with Claude:

Hallucination risk: Low

Explanation: The combination of a vector database and RAG offers the best chance of minimising hallucinations. The vector database enables more accurate and relevant information retrieval based on semantic similarity, reducing the chances of missing important context. The RAG technique ensures that the retrieved information is tightly integrated with the generation process, keeping the model grounded in the factual information stored in the vector database. This approach allows for more control over the information the model uses to generate responses, minimising the risk of hallucinations.