AI Based Search using SearXNG, AWS Bedrock and Perplexia

AI-based search has revolutionized how we access and interact with information online. By leveraging advanced algorithms. These systems utilize machine learning and deep learning techniques to analyse vast amounts of data, improving accuracy and efficiency in retrieving information34. Additionally, AI-driven search capabilities extend beyond traditional web searches, influencing various applications including e-commerce, social media, and content streaming platforms by tailoring experiences to individual users5. As AI continues to evolve, its impact on search technology promises.

Amazon Bedrock is a fully managed service from AWS that simplifies the development of generative AI applications by providing access to a variety of foundation models (FMs) from top AI companies, including Anthropic's Claude 3 Sonnet and Amazon's Titan models. This platform allows developers to experiment with, customize, and integrate these models into their applications without the need for extensive machine learning expertise or infrastructure management.

The Claude 3 Sonnet model, showcases significant advancements in speed and reasoning capabilities, making it suitable for enterprise-level generative AI applications. It is designed to handle complex instructions effectively and can evaluate generated content, providing feedback and suggestions for improvement, thereby enhancing user experience.

Additionally, the Amazon Titan embedding model offers optimized retrieval augmented generation (RAG) capabilities, enabling users to enhance their applications with up-to-date information and tailored responses. This integration of Titan embeddings with Claude 3 Sonnet allows for a more robust and versatile approach to developing AI solutions that meet specific business needs.

In this blog we will be covering implementation of AI Based Search using Open-Source tools searxng, Perplexia with AWS Bedrock based models, in this implementation we have hardcoded for AWS Bedrock Anthropic Claude sonnet 3.0 model for LLM and Amazon Titan model for embedding.? We will be modifying the code later to add support for other LLM as well.

We will be using the Open-source project “Perplexia” implemented by “Kushagra”. The entire backend service has been modified to use Bedrock chat and embedding libraries.

We will not be covering the steps to create ec2 instance. we will assume this is already available. We will start from associating an Elastic IP to the EC2 instance and installing the required software’s.

?Prerequisites

·???????? Security Groups

·???????? EC2 instance

·???????? AWS bedrock Console.

·???????? Access to the required Foundation Models

·???????? AWS Access & Secret Access keys with permission to AWS Bedrock related Policies.

?

?Walkthrough

The code has been fully modified to support Bedrock. All usage of Chat OpenAI has been replaced with Bedrock related Libraries.

The “config.toml” file has the following settings

?We need to configure the AWS user and for that user we need to grant Bedrock full Access privilege and use that Access Key and secret Access Key. Also, in Bedrock console we need to make sure we have access to all the Models required.

Let us first create a IAM user, give permission to access bedrock models and generate Keys to be used.

For the user we created Please associate the following policy

Next, we will generate the Keys

Press “Create Access Key” and save it securely. This should not be shared.

Press “Next”.

Press “Create Access Key”.

Press Download .csv file.

?In the Bedrock console, request and get access for Bedrock model (Anthropic Claude sonnet 3.5).

In AWS Bedrock -> Model Access.

We will start with EC2 instance and associate an Elastic IP to the instance.

Steps to install Docker.

?Steps to Install Docker compose.

?

?Steps to Install Make in Amazon Linx2 instance.

Steps to install git in Amazon Linx2 instance.

Steps to Install Searxng for AI Search engine.

?The EC2 instance is running. Now let us assign an elastic IP for the same.

?

Press “Allocate”.

Now associate this to the EC2 instance

Press “Associate”.

We have done the first part.Let us connect to the instance via putty.

Let us install Docker and docker-compose

Next docker-compose.

Next, we will install the make and git in Amazon linux 2

Next, we will install Git

Next, we will create a virtual environment and deploy Searxng.

?Create a Virtual environment.

Load required packages.

Git clone the searxng source

under searx directory in settings.yml file, change following:

search:

? formats:

??? - html

??? - json

???

Build the image.

Deploy the container and test.

Get the private IP address associated with the EC2 instance.

docker run --rm -d -p 8181:8080 -v "${PWD}/searxng:/etc/searxng" -e "BASE_URL=https://172.31.21.1:$PORT/" -e "INSTANCE_NAME=my-instance" searxng/searxng

We will test the Search engine.

https://44.198.108.129:8181

?

This is up and running. Next, we will configure Perplexia and demonstrate AI based search using AWS Bedrock Anthropic Claude sonnet 3 model.

In the Folder Perplexia we will have the file config.toml file make the changes in the file as shown below.

We will run the entire search engine using Bedrock models and Bedrock libraries. Then build and run the backend and UI.

Starting the backend and UI services.

For the backend

For the UI.

?

The Model configuration used.

?Let us open https://localhost:3000 from browser.

Ask the First Question

?

?

?

Cleanup

Stop all the services in ec2 instance and terminate the instance. Remove the AWS Keys used for this demo for additional security.

?