Title: RAG on day-2(need for Agentic RAG)

Day-0:

Language models without the ability to process proprietary data are not useful for most of the business application.

Day-1:

RAG(Retrieval augmented generation) was a pretty smart way to consume proprietary data with exposing them to the world( we still don’t know if this is fully true with cloud hosted models) but yes they helped the organizations moved forward towards trying out wit POC. For most of the organizations the first use that crossed their mind was to improve productivity by making democratizing information availability internally (to employees) and externally (to customers).

This was day-1 and a lot of organizations have developed their custom solutions or with managed services in the cloud.

Day-2: What’s next?

While RAG can help you find relevant data “most” of the time, the first major challenge with simplified RAG implementation is ability to perform a slightly complex problem. One simple explain where a linear RAG pipeline will fail is when you want to summarize the document and perform vector search using the same pipeline.

Secondly if you have internal scripts which can generate some context which can then be provided as additonal context while synthesizing a response.

Thirdly, the ability to have long and/or short memory for making the conversation meaningful.

So what should be Day-2 ?

Its worthwhile to start exploring agentic implementations for RAG. Simplified version of agents can provide Tool use(or function calling)which is the ?ability to call your scripts to integrate your business application/tools into the pipeline and more advanced implementation of agents can includes multiple agents with some orchestrator which can perform dynamic query planning ?and assign specific tasks to different available Agents.

One of the most simplified version of multi agent implementation will be using a routing to either summarize or provide context based synthesized answers.

Needless to state agentic Implementation will be complex however many orchestrator tool like Langchain, Autogen and crew.ai provides simplified abstraction to provide a reasonable jump at the start.