Optimizing Healthcare Solutions with Advanced RAG

Transformative Synergy: Optimizing Healthcare Solutions with Advanced RAG

In the dynamic sectors of insurance and healthcare, access to accurate information is crucial. Traditional methods usually struggle to give precise and relevant answers. As more people turn to digital platforms for their healthcare questions, the demand for dependable Q&A systems is becoming increasingly important.

In this blog, we delve into how we used the revolutionary approach known as RAG to build a healthcare Q&A system. We will walk you through our journey from a basic RAG setup to a more advanced version. We will discuss how this upgrade helped us deliver accurate, comprehensive and precise answers. Join us as we explore the transformative power of Advanced RAG in healthcare Q&A!

Application of Advanced RAG Approach in Healthcare:

One of the most significant challenges faced by consumers is navigating the complex landscape of insurance options. With a multitude of plans, coverage details, individuals often struggle to find the insurance that best suits their unique needs. This complexity is compounded during enrollment periods when making the right choice is crucial.

To address this challenge, there is a pressing need for a personalized insurance recommendation Q&A system tailored to the healthcare industry domain.

The healthcare insurance market offers a vast array of plans with varying coverage, benefits and costs. Generative AI (Gen AI) and Large Language Models (LLMs) can process and understand vast amounts of data, making it easier to parse through complex insurance information and provide accurate recommendations.

Each individual's healthcare needs, and financial situations are unique. Gen AI/LLM can analyze individual data points and tailor recommendations to suit personal variables.

We worked with the below datasets for creating the Q& A system for healthcare Insurance.

Datasets available with us:

• Healthcare Policy and Coverage Documents

This dataset outlines the terms and conditions, benefits, and limitations of a health insurance plan. These include key elements such as – Policy overview, covered services, exclusions and limitations, cost sharing details, provider networks, claims procedures, and policyholders’ rights and responsibilities.

Below are the characteristics of this dataset.

Number of Pages : ~200

Characteristics : Nested Tables, Plan comparison tables, Nonstandardized layouts etc.

Domain Terminologies : Co-pay, In-Network, PPO, health risk assessment (HRA), durable medical equipment (DME)…

Healthcare Rider plans

This dataset contains information about additional insurance coverage options known as riders, that can be added to a primary health insurance policy i.e.,

• Dental
• Vision
• Critical Illness
• Maternity
• Accidental Death Coverage

It outlines the specific benefits provided by each rider, the associated premiums, coverage limits, exclusions, and eligibility criteria. This information aids in customizing the insurance plans to better meet the diverse needs of policyholders.

Drug Formulary

This dataset contains a list of prescription medications approved for use within a particular health insurance plan. The dataset includes detailed information about each drug, its generic and brand names, dosage norms, any formulary restrictions or tiered coverage

levels that determine patient access and copay amounts etc. in a ~200 pages pdf document.

Customer Demographics

This dataset contains detailed information about the characteristics of individuals or a corporate employee. It typically includes –

• Basic demographics – Age, gender, marital status etc.
• Socioeconomic status – income levels, occupation, employment status etc.
• Geographic information – residential addresses, zip code, regions etc.
• Behavioral information and Preferences – lifestyle, alcohol consumption, smoking status etc.

This dataset helps in tailoring services to meet specific needs of demographic groups.

We created the first iteration of the Q&A solution using simple RAG approach.

Let’s understand briefly the RAG approach.

What is RAG (Retrieval-Augmented Generation)?

RAG optimizes the output of a large language models (LLMs) by combining retrieval and generation techniques. This approach makes generated text more accurate and relevant without needing to retrain the model. RAG tailors the model for specific domains or additional knowledge bases, enhancing tasks like Q&A systems, conversational AI, and content generation. We can say, RAG has become a game-changer for making natural language processing (NLP) applications more effective.

The RAG approach is a two-step process that leverages both retrieval and generation to provide high-quality information:

1. Retrieve: The approach first searches through large datasets, knowledge bases, or documents to find the most relevant pieces of information in response to a query.
2. Generate: Using the retrieved information, it then generates a coherent and contextually appropriate response.

• Post implementing using simple RAG approach, we observed few shortcomings in the answers. In some cases, the response was not comprehensive. Looked like certain points are missed though those are in vector store.
• Accuracy was not as desired. Hence, it was unable to produce reliable results for health care data (complex data on Insurance Coverage)
• Certain inconsistencies observed in the response Here is an example of the response post implementing simple RAG approach –

To provide answer to below query requires understanding of the knowledge spread across different parts of the document. The top-K retrieved short contiguous texts or chunks will not contain enough context to answer the question. When we evaluated the response, we noticed that simple RAG was unable to retrieve key information present in the document to answer the question.

The correct response should have been -

To boost response quality, we started exploring advanced RAG methodologies. After experimenting with a range of techniques, we selected and combined the ones that best fit our use case.

Let's take a quick look at the techniques we chose and how they enhanced our results.

Advanced RAG

Query Decomposition, Function Call

To further enhance LLM capability, we employed advanced techniques like Query Decomposition and Function Calls to help handle complex queries more effectively and accurately, ensuring users receive the most relevant information and recommendations.

Query decomposition involves prompt engineering to instruct LLM to break down a complex query into simpler or more manageable sub-queries. This process ensures that each component of the query is addressed accurately, improving the overall quality of the response.

Here are the steps involved in Query Decomposition:

Query Understanding: Analyzing the initial complex query to grasp its full context and intent.

Sub-Query Generation: Splitting the complex query into smaller, more focused subqueries.

Individual Query Processing: Processing each sub-query independently to retrieve precise information.

Aggregation: Combining the results from all sub-queries into a cohesive, comprehensive response.

Let’s see one example on how the query decomposition worked:

Subqueries generated by LLM:

1. “What are the benefits of a high-deductible health plan for reducing out-of-pocket costs?”
2. “How do health savings accounts (HSAs) work as a cost-saving alternative to traditional h ealth plans?”
3. “What are the differences between preferred provider organizations (PPOs) and health maintenance organizations (HMOs) in terms of out-of-pocket costs for clients?”

The multi-query in most use cases can provide a more complete response, addressing all the topics involved in the base query. We observed in our case the Multiquery approach proved to be helpful in cases with complex and long questions.

We also made use of Function calling to dynamically invoke specific functions or APIs to fetch additional information from external knowledge sources to tailor response to the user queries.

RAPTOR (Recursive Abstractive Processing of Tree-Organized Retrieval)

RAPTOR model allows us to have holistic understanding of the overall document context and retrieve information across long tail documents.

Using the RAPTOR approach, we first generate embeddings for the health care policies and drug formulary texts, clusters them based on similarity, expand the cluster assignments for easier processing and then summarizes the content within each cluster.

This tree structure is essential for RAPTOR's function because it captures both high-level and detailed aspects of text. This capability is particularly useful for handling complex thematic queries and multi-step reasoning in Q&A tasks.

This technique allowed us to overcome the limitation of simple RAG approach where only few short contiguous text chunks are retrieved, losing out on the information present in multiple parts of the document.

RAG was not able to provide desired answers for holistic kind of questions when we embed long-tail domain knowledge. Hence, we opted for RAPTOR methods.

Conclusion

Depending on the use cases, complexity of queries, data (information spread across documents, large context, document structure), we can apply hierarchical or chunked retrieval mechanism that can process and prioritize relevant sections of the context. By integrating various advanced RAG techniques, we can develop systems that can understand the intricacies of user queries and deliver personalized recommendation with accuracy. We have also applied HyDE technique which we will cover in our next blog.

Reference

[1] Advanced RAG for Enhanced and Optimized Results | CoffeeBeans

[2] RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval (arxiv.org)