RAG Document Search Done Right

Sarah is a senior enterprise applications manager at a large multinational corporation, tasked with leveraging the latest technologies on the market to improve access to information for business users. Company knowledge is buried deep in many thousands of documents in the company intranet. Finding anything is extremely difficult unless you know the correct magic phrase to enter into the traditional search engine. Sarah hears about Retrieval-Augmented Generation (RAG) and Large Language Models (LLM) as advanced new technologies that might help solve the problem. She starts a project to build a prototype.

After months of work, the prototype is ready. It’s initially impressive, but very quickly workers start complaining. The tool is: too slow, inconsistent, buggy, hard to update, misses key documents, and no one trusts anything it says. Disillusioned, Sarah cancels the project. Is Generative AI overhyped?

Understanding RAG

When a user asked a question, RAG search uses semantic search to find relevant information from a large corpus of unstructured data. It then augments a Large Language Model prompt with that information to provide a natural language answer. It differs from traditional keyword search because it can find information related to a user’s question, even if none of the words in the question appear in the indexed documents.

Why is it not straightforward?

Datch offers a mature RAG-powered search, enhanced by many improvement iterations. Sarah’s prototype was missing many key refinements:

Understanding Images and Parsing Tables: Beyond text, many documents contain valuable information in images and tables. A robust RAG system must be capable of interpreting and extracting data from these non-text elements to provide comprehensive search results.

Expanding User Queries: Users are used to keyword search. Semantic search requires users to learn how to write their queries in a way that semantic search can find the correct information. A good RAG system should be able to expand and enrich a user’s keyword query, understanding the intent behind the keywords and rewriting the query to provide the most relevant results.

Switching Between Full-Text and Semantic Search: Depending on the query, either traditional full-text search or semantic search may be more appropriate. An intelligent RAG system can dynamically switch between these methods based on what will return the best results.

Drag and Drop Document Upload: Ease of use is critical. A user-friendly interface with drag-and-drop document upload capabilities ensures that users can easily add new documents to the system without technical barriers.

Consistent Search Results: Consistency is key to user trust. A well-designed RAG system should deliver consistent answers when the user asks the same question.

Removing Hallucinations: LLMs tend to make up convincing incorrect information if they don’t know the answer to a question. Users naturally lose trust in such systems if they cannot trust the answers they provide. A well-designed RAG system mitigates the hallucination problem by ensuring answers are always based on the information in the source documents.

Handling Multiple Versions of Documents: A company often has multiple versions of documents. The system must be able to recognize and handle different versions, ensuring users always access the most relevant and up-to-date information.

Handling Contradictory Information: Inconsistent or contradictory information across documents is a common challenge. A sophisticated RAG system can identify and inform the user about contradictory information, allow them to make informed decisions.

Provenance: To be trusted by users, a RAG search system must provide provenance for every answer it gives. It must reference the documents and pages that its answers are based on and make it easy for the user to verify the answers.

Speed: Semantic search and LLMs can be slow to run. They are new technologies that are significantly more complex than traditional search indexes. A good RAG system is optimized for speed, so users don’t have to wait for the answers to their questions.

Continuously Optimizing Search Results: A RAG system cannot just be deployed and forgotten about. To achieve the best search performance, it is crucial to work with domain experts to fine-tune the search. Every time the search delivers a wrong answer, experts can find the correct answer and update the system to learn from its mistake.

Datch’s RAG Document Search

At Datch, we have a ready-made solution that can quickly deliver value to a business. We have taken all the above nuances into account and built an enterprise-ready solution. Datch will also work with a client to fine-tune the results of the search.

Interested in seeing a demo of how this work for yourself? DM me or let us know in the comments section.