AWS Bedrock Anthropic Claude based Guardrail enabled RAG with Kendra, AWS Knowledge Base and AWS Neptune as Data store
Satish Srinivasan
Cloud Architect I Cloud Security Analyst I Specialist - AWS & Azure Cloud. AWS Community Builder| AWS APN Ambassador
Generative AI is a type of artificial intelligence technology that can create new content, such as text, images, code, or music, based on existing data and patterns. Generative AI (GenAI) and large language models (LLMs), such as those available via Amazon are transforming the way developers and enterprises can solve traditionally complex challenges related to natural language processing and understanding. Some of the benefits offered by LLMs include the ability to create more capable and compelling conversational AI experiences for customer service applications and improving employee productivity through more intuitive and accurate responses.
To restrict the GenAI application responses to domain specific data only, we need to use a technique called Retrieval Augmented Generation (RAG). An application using the RAG approach retrieves information most relevant to the user’s request from the enterprise knowledge base or content, bundles it as context along with the user’s request as a prompt, and then sends it to the LLM to get a GenAI response. In designing effective RAG, content retrieval is a critical step to ensure the LLM receives the most relevant and concise context from enterprise content to generate accurate responses.
Guardrails in the context of Generative AI refers to guidelines implemented to ensure the responsible and secure usage of Large Language Models (LLMs) in organizations. These guardrails serve as a framework to shape the behaviour of AI models, allowing enterprises to build generative capabilities on a foundation of trust. Guardrails help organizations govern and constrain end-user interactions, ensuring compliance with enterprise policies and ethical guidelines. They are essential for preventing the generation of inappropriate, biased, or misleading content that could harm an organizations reputation or violate legal obligations.
By leveraging these Guardrails, we can implement safeguards tailored to the application requirements and responsible AI policies, while ensuring a consistent level of AI safety across all generative AI applications.
We will demo the above approach using Various AWS data sources (Kendra, AWS Bedrock knowledge based, OpenSearch serverless & Neptune) in giving the user flexibility to select the best one based on their business needs.
Amazon Kendra is an intelligent search service that leverages machine learning and natural language processing to provide precise answers to queries across various data sources. It enhances enterprise search by understanding context and semantics, allowing users to efficiently retrieve relevant information from unstructured data repositories. Amazon Kendra enhances Retrieval Augmented Generation (RAG) applications by providing an intelligent search capability that retrieves the most relevant documents from an indexed knowledge base, ensuring high-quality context for generative AI responses. Its machine learning-powered search features allow for semantic understanding and precise document ranking, which significantly improves the accuracy of answers generated by large language models (LLMs) in RAG workflows.?
AWS Knowledge Base is a fully managed capability that enables organizations to implement the Retrieval Augmented Generation (RAG) workflow, allowing them to integrate proprietary data sources for enhanced AI responses. It automatically fetches, processes, and stores data as embeddings, facilitating efficient querying and contextual augmentation for generative AI applications. This used Amazon OpenSearch Serverless Behind the Scenes.
Amazon OpenSearch Serverless is an on-demand, serverless configuration for Amazon OpenSearch Service that automatically provisions and scales resources based on application needs, eliminating the need for manual management. This setup allows users to efficiently search and analyse large volumes of data without the complexities of managing OpenSearch clusters.
Amazon Neptune is a fully managed graph database service designed for high-performance applications that require efficient storage and querying of highly connected datasets. It supports popular graph models and query languages, enabling use cases such as fraud detection, knowledge graphs, and recommendation engines. Amazon Neptune facilitates Retrieval Augmented Generation (RAG) applications by enabling vector similarity searches through its graph database, allowing for efficient retrieval of relevant contextual information from dense data representations. This capability enhances the accuracy and relevance of generative AI outputs by leveraging knowledge graphs to relate concepts and entities, thereby improving the overall quality of responses generated by large language models (LLMs) in RAG workflows.
Architecture overview
The Architecture diagram for Bot using AWS Bedrock Anthropic Claude Model and the various data source.
Prerequisites
AWS Bedrock, EC2, Kendra, Open search serverless, Knowledgebase and Neptune serverless & Notebook. EC2 instance will be used for deploying the streamlit App.
?
Walkthrough
First let us create a S3 bucket and upload the Domain data which we will be using for this Generative AI bot.
Creating Guard rail which we will be using in the RAG bot.
Press “Create guardrail”.
Press “Next”.
Press “Next”.
Let us create three denied custom topics.
Press “Confirm”.
Next, we will add topics for War related, Race religion and caste
Press “Confirm”
Press “Confirm”
Press “Next”.
Enable Profanity Filter and press “Next”.
Press “Next”.
Press “Next”.
Press “Create guardrail”.
The guard rail is ready and can be integrated with the RAG bot.
RAG Bot with AWS Bedrock Anthropic Claude 3 sonnet and AWS Kendra as data store.
Create Kendra Index
Press “Create index”.
Press “Next”.
Press “Next”.
Press “Next”.
Press “Create”.
The Index is getting created. Wait for it to be available.
The Index is ready, now let us add the data source , this is the pdf file we showed in the beginning.
Press “Data Sources”.
Select the Amazon S3 connector.
Press “Next”.
Press “Next”.
The Bucket where we stored the pdf file.
The folder in the bucket where we have the file.
Press “Next”.
Press “Next”.
Press “Add data source”.
The data source is not ready. Press “Sync now” to update the index with the data in the PDF file.
Wait for this to be completed before we proceed.
The data has been loaded. Now we will run the RAG bot.
This is the ID we will use for Kendra Index in the below file.
Configure the .env file for the Kendra Index and Guardrail identifier.
We will read the values dynamically in the Bot.
Similarly for Kendra also we will read the value dynamically.
We will associate the guardrails with the Bedrock API as shown below.
The same with no guardrail is as shown below.
Now we will run our Bot and demo the RAG feature with Guardrails.
Let us test the Bot by asking questions.
We got the Answer. Next, we will ask the same question but add a profane word along with the Question.
Next, we will ask a Question on the denied topics.
We can keep adding Topics which are customized to our business needs to stop answering unrelated questions. Also, we can add custom profane words to make our RAG Bot more secure.
Next, we will demo the same using AWS Bedrock knowledge base as data source.
RAG Bot with AWS Bedrock Anthropic Claude and AWS knowledge base as data store.
Create AWS knowledge base.
Press “Knowledge bases”.
Press “Next”.
Press “Next”.
Press “Next”.
Press “Create knowledge base”.
Wait for it to complete.
?
We need to go to the database section and run the sync job to load the data to the Knowledge base index.
The knowledge base is created. We will be using this as source for our RAG Bot.
Content of the .env file
Setting the configs in the Bot
Retrieving the data from the Knowledge based on the user query
Starting the Bot
Let us Test the Bot now.
领英推荐
Start asking questions.
?
Next, we will demo the same using AWS Neptune serverless as data source.
RAG Bot with AWS Bedrock Anthropic Claude and Neptune serverless as data store.
Create a security group for Neptune DB.
Let us create a Neptune Serverless Graph database.
Press “Create database”.
?
We will be creating the notebook manually.
Press “Create database”.
Wait for the status to be Available.
Next, we will create the Notebook.
Press “Create notebook”.
Press “Create notebook”.
Wait for it to be available.
The Notebook is ready.
Press “Open JupyterLab”.
Press File->New->Notebook
We will continue with Python3 Kernel.
Type “%stats” to check the status of the DB instance.
The Cluster is healthy.
Next, we will load the sample data. I will be sharing this along with the blog. This is a Movie database in Graph format.
The data is loaded successfully.?
Let us run a query to check for one set of records from the DB.
?There is data in the DB. Next, we will add policies to the Role we created as we will be using this for other access.
Press “Add permission”. For this demo we are not considering Least privilege.
Let us install the required packages
Let us now create a connection.
Let us Query the Graph DB.
We see the Query and the O/p of the query. ?Next, we will run the same using Anthropic Claude 3.
Next, we will do the same search using streamlit app.
First Question
?Second Question
In the next part we will parse a pdf file and convert it to graph format and load the Neptune DB. Once it is Loaded, we will query the DB and do a demo of knowledge graphs.
?Packages to be installed.
Converting pdf to graph data and storing it in Neptune DB. For embeddings we will use titan model.
The files are under the folder “data.”
Query the Graph database.
Check the response.
This completes the knowledge Graph using AWS. Next we will cleanup the resources we created as part of this demo.
Cleaning Up
Stop and delete the EC2 instance.
Cleaning AWS Kendra
?
Press “Delete” to delete the data source.
Type “Delete” and press “Delete”.
Press “Delete”.
Wait for this to be completed before we delete the Index.
The data source has been deleted. Next we will delete the Index.
Press “Delete”.
Press “Delete”.
The Index has been deleted.
?Cleaning Knowledge base
First Go inside the knowledge base and delete the Index, else we will be charged for it even after deleting the knowledge base. We can also check for the same under AWS Open search as shown below.
Go inside the knowledge base and go to the Data source section
Press “Delete”.
Press “Delete”
The Data source is deleted. Next, we will delete the Knowledge base.
Press “Delete”.
Press “Delete”.
The knowledge base has been removed. Next, we will delete the associated vector store.
Cleaning Neptune DB and the Associated Notebook.
First we will stop the Notebook instance
Press “Stop”.
Press “Stop”.
Wait for it to stop before we delete that.
Now we can delete the Notebook
Press “Delete”.
Press “Delete”.
Wait for this to complete.
Next, we will delete the Neptune Cluster
Press “Delete instance”.
Press “Delete instance”.
Wait for this to complete.
The Notebook has been deleted.
The Cluster has been deleted.
Conclusion
In this Blog, we learned how to develop a RAG Bot using Anthropic Claude with Kendra, AWS Knowledge Base and Neptune (Graph RAG). In the next series we will see how we can extent this to videos and search on Video chapters.
?