Engineering the AI applications (Part 7): Conversational AI (RAG) Applications

In this short post, I would like to share a few lessons that we have learnt while developing the AI applications.

Developing a Chat applications is not new. Having seen the chat applications from IRC to XMPP to ChatGPT, I believe a new era is in the offing for chat applications.

Certainly IRC, XMPP and Websocket protocols are backbone of chat applications. The protocols are mature and pervasive as well. However, taking advantage of AI is a big leap to Chat applications.

What is RAG (Retrieval Augmented Generation)?

RAG is a natural language processing (NLP) model that combines both retrieval and generation techniques.

About RAG: Start with https://arxiv.org/abs/2005.11401

One size doesn't fit all

It is very challenging if an organization just depends on one service provider. It means, organizations need to develop in-house capabilities to "assemble" AI enabled Chat solutions to be agile in the market.

Assembling AI Enabled Chat Solutions:

Out of several options, we have considered the following solution stack based on Open-source (self-hosted) with a few exceptions.

• Bot Front-end: Typebot (https://typebot.io/). This allows building a complex chat-flows and integrate them to the AI backend. It has very clear separation of flow-builder and viewer. Typebot may be replaced with IRC or XMPP bots.
• Workflow Engines: n8n (https://n8n.io/), Automatisch (https://automatisch.io/), Node-RED (https://nodered.org/)

Bot Front-end <--- Integration --> Workflow Engines

AI Services (+Agents): This is a hard part! AI services shall have Generative AI capabilities AND Classical AI capabilities AND Machine Learning capabilities.

• FlowwiseAI (https://flowiseai.com/). This provides a convenient way (APIs) to connect with the AI backend (ChatGPT, LocalAI etc.).

Bot Front-end <--> Workflow Engines <---> AI Services (+Agents)

• Supabase (https://supabase.com/) as Vector DB. There are several alternatives such as Pinecone (https://supabase.com/) etc.
• LocalAI (https://localai.io/)
• Huggingface (https://huggingface.co/)
• OpenAI (https://openai.com/). No doubt it fuels the next generation conversational applications.
• Picominer ( Vidcentum Technologies ): a collection of open source libraries, frameworks, languages for classical AI/ML application development. It includes IoT, data processing, rule engines, telemetry, gateways and mobile applications.

Application Builder:

Appwrite (https://appwrite.io/) brings several features to build apps with ease and make development easier.

Summary:

In order to offer context-ware content services to the customers, it is essential to acquire the new capabilities of AI and integrations. The above solution stack is one of the possible "assembly lines".

If you would like to know more about our approach, drop a message to Maruthi Pathapati or in the comments.

Affiliations

I write posts for Vidcentum Technologies where I am founder Director. Some of the articles are published in my blogs / articles / posts as well as on the company timeline / posts.