Retrieval Augmented Generation RAG - An Approach from going RAGS to Riches for Enterprises

We have all heard of the term ‘Rags to Riches’ and keep hearing about such inspiring stories about individuals who truly transformed their lives, coming from very humble backgrounds to reaching soring heights of success, be it in the field of technology, business, or political arena. While the journeys of each will be different and the paths followed may have had a different trajectory. One of the things common to all would have been the ability to apply the knowledge and make use of ‘Information’.

As they say Information is Power, thus the information which is accurate, timely and trusted can make difference between the have’s and the have nots, in terms of opportunities being grabbed.

With the democratization of technology, the key distinction for an Enterprise to be able to augment its PPT (not Powerpoint) but People, Processes, & Technology) will be to get access to the most trusted and relevant information, leveraging historical data of organization, mapped it with the world’s present data, to be presented in a manner which can be easily used and trusted to make important decisions.

The GenAI technologies like ChatGPT are not just transforming but disrupting our corporate way of working, improving it manifolds, and will be becoming part of our habits, on how we consume information, use information and apply it our day-to-day activities.?

While ChatGPT and other similar GenAI techs like Google Gemini AI or Meta’s LLaMA are general purpose LLM (Large Language Model), which are effectively being use by millions and billions of people, the information generated by these cannot be trusted for ready applications when it comes to enterprise decision making.

Typical challenges of using the GenAI, can be picked? from other article on #accountableAI https://www.dhirubhai.net/pulse/accountable-ai-lets-talk-kamaljeet-kj-singh/

Corporates require timely and accurate information, which if augmented with AI can bring about tremendous benefits for the organizations. These benefits may range from

·??????? improving workforce efficiency,

·??????? faster & improved research & development,

·??????? shorter product lifecycles,

·??????? reduced errors in planning,

·??????? better targeted customer reach etc.

A typical solution ask that all enterprises have is,

·??????? how can it use the GenAI capabilities to be trained on its data,

·??????? whereby it understands its organization,

·??????? knows about the organization history and help it find information through the vast Terabytes and petabytes of data,

·??????? organize it in a manner that any user/employee can access it easily, wherein the individual can apply his/her skills to enhance it to generate information needed for his/her work

To solve this problem a framework/architecture called RAG (Retrieval Augmented Generation) was devised for retrieving information through #NLP Natural Language Processing, querying through the companies Data store, further augmenting it with GenAI capabilities of Transformer models which are General purpose LLMs, ?to provide most relevant and accurate information for the individual.

The architecture was designed to limit the shortcomings of a General Purpose LLM like

·??????? hallucinations or inaccurate information

·??????? lacking domain data

·??????? higher infrastructure compute requirements, which implies higher costs

·??????? risks of exposing personal or private information

and get more targeted response, enriched with the domain knowledge/information.

?A basic automated RAG model primarily will have 3 components

1.????? A UI (User Interface) for collecting the input from User and displaying the output.

2.????? A Retrieval block, which would be like search capabilities which will have search information from the knowledge based or information storehouse

3.????? A Generation Block, which will basically be taking the User Input and output of retrieval block to generate an output which is essentially augmented and not merely displays the search output.

?

?

While the above functional architecture may seem oversimplified for a complex use case like enterprise search and retrieval, it has become a standard norm in terms of organizations aiming to implement a Knowledge search based on GenAI capabilities. ?

There are many other aspects of consideration that an organization will have to take into account, after all it is a matter of going from RAGs to Riches and if not done properly can bring an enterprise to rags position (not literally)

?

Some of the critical aspects to be considered while implementing RAG

1.????? Data privacy – How much and what all data to be made available in the Knowledge Bank. This would also imply what data as an outcome of search can go to Public LLMs like Gemini or ChatGPT

2.????? Data Access – while enterprise may aim to bring the power of knowledge of search to every individual, it can pose risks of exposing confidential information to personnel which may not require access to such information.

3.????? Biases - Influence of biases in the generated information from the knowledge store, coupled with ?biases in the Public LLMs, can impact the decision-making process

4.????? Relevancy- an un trained model can still end up retrieving output based on historical data which may not be relevant in the terms of the summaries which get generated

5.????? Data Accuracy and Ordering – an untrained model may not present an accurate output, just as LLMs were trained on vasts amounts of data, ?a RAG must have a training module to improve the accuracy of output and if the expectation is of implementing a Search Use case, then it must have some capabilities to rank the output

6.????? Contextualization- Most Search engines as we know contextualizes the experiences for individuals by bringing the out most relevant, latest and suitable information to the user, based on his/her past searches and other activities that are performed on the web. The RAGs will have to have an element to contextualize outputs based on roles and user personas within the organization.

7.????? Training of staff – Just as there are people who can use Google Search to pull out most relevant information faster compared to other, based on their experience of using different search terms, the staff will have to be trained to write proper search prompts.

?

Thus a slightly more complex functional architecture which can address some of the concerns mentioned above will be with the introduction of

1.????? Data and Access Management – to ensure principles data access management are implemented.

2.????? User Persona and Context Management – a module which will ensure the most relevant, accurate and contextual information is presented, based on the query

3.????? Prompt Engineering – to improve the accuracy of LLMs when it comes to an Enterprise user behavior and requirements

4.????? Content Moderation – this can be a combination of Human in the Loop (HITL) or intelligent trained module which can limit, flag and alert of any sensitive information is requested by a user.

When it comes to implementing a RAG architecture, the approach may vary depending on the priorities, urgencies and listing of the use cases. The general approach to be followed and maturity of an organization can be referred from the following graph.

I hope this information would be helpful for enterprises planning to embark on the journeys of implementing RAG or GenAI use cases in their organizations. I am fortunate to be part of the teams which are building such complex solutions & products and taking them to the market.

?

Cloud and Open Source technologies with Open standards has made it easier for organizations to quickly adopt such advanced technologies and drive better outcomes to stay ahead of the competition.

?

Looking forward to learn more on how RAG is being implemented in your organization, challenges faced and lessons learned.

?

References

https://www.ibm.com/architectures/hybrid/genai-rag

https://learn.microsoft.com/en-us/azure/search/retrieval-augmented-generation-overview

https://www.databricks.com/glossary/retrieval-augmented-generation-rag

Cover Image source - https://www.freepik.com/vectors/money-mountain