Local RAG Chatbot: Capabilities & Setup Guide

?? Capabilities of This RAG System

This Retrieval-Augmented Generation (RAG) Chatbot provides the following functionalities:

? Document Upload & Parsing → Supports PDFs & DOCX files for knowledge extraction.

? Question Answering → Users can ask questions based on uploaded documents.

? Local AI Processing → Runs Mistral/Llama3 models locally via Ollama (no external API calls).

? Vector Search → Stores document embeddings using ChromaDB for fast retrieval.

? Simple Web UI → Built with Flask & Bootstrap, making it easy to use.

?? Preparing the Environment

System Requirements

• macOS/Linux/Windows
• Python 3.8+
• Virtual Environment (Recommended)

Install Required Dependencies

Run the following commands to set up your environment:

# Create and activate virtual environment
python3 -m venv rag_env
source rag_env/bin/activate  # On Windows use `rag_env\Scripts\activate`

# Install dependencies
pip install flask flask_cors chromadb \
            langchain-community langchain-huggingface \
            unstructured[all] pypdf python-docx        

If using Zsh, escape square brackets:

pip install "unstructured[all]"        

?? Step 1: Set Up Flask Backend (app.py)

Create a new file app.py and paste the following code:

This is a Flask-based web application that serves as a Retrieval-Augmented Generation (RAG) chatbot. It allows users to upload PDF and DOCX files, processes them into a vector database (ChromaDB), and enables question-answering through a locally running DeepSeek R1 14B model via Ollama.

1?? Initializing the Flask App

• The application uses Flask as the web framework.
• A folder named "documents" is created to store uploaded files.

2?? Loading and Processing Documents

• The app scans the documents folder and processes any PDFs or DOCX files.
• If a file is a PDF, it is loaded using a PDF processor.
• If a file is a DOCX, it is loaded using a Word document processor.
• All document content is then split into chunks to optimize retrieval performance.

3?? Creating a Vector Database

• The text chunks from documents are converted into vector embeddings.
• These embeddings are stored inside ChromaDB, a high-performance vector database.
• The embeddings are created using a sentence-transformer-based model.

4?? Setting Up the Language Model

• The Ollama framework is used to run a DeepSeek R1 14B model (or Mistral, Llama3).
• The model is configured to retrieve relevant document chunks when answering questions.

5?? Defining Web Routes

The application defines several Flask API routes:

• Homepage (/)
• Upload Endpoint (/upload)
• Question Answering Endpoint (/ask)

6?? Running the Flask App

• The application runs on port 5001 and is set to debug mode for easier development.
• The web app can be started using a standard command.

?? Step 2: Set Up the Frontend (templates/index.html)

Create a folder templates/, then inside it, create index.html:

Structure of the Web Page

• The page follows a standard HTML5 structure, ensuring compatibility with all modern browsers.
• The Bootstrap framework is used for styling, improving responsiveness and usability.

2?? Web Page Title & Metadata

• The page is titled "RAG Chatbot UI".
• It sets meta viewport settings to ensure a mobile-friendly layout.

3?? Including Bootstrap for Styling

• The page loads Bootstrap 5.3 from a CDN to provide: Pre-styled buttons, forms, and layout grids. A clean and user-friendly UI.

4?? Main User Interface Elements

?? Document Upload Section

• Label: "Upload Document (PDF/DOCX)"
• Input Field: Allows users to select a file.
• Upload Button: When clicked, triggers the file upload function.

? Question Input Section

• Label: "Ask a Question"
• Text Input Box: Users enter their questions here.
• Ask Button: When clicked, it sends the question to the RAG chatbot.

?? Answer Display Section

• Heading: "Response"
• Text Container: Displays the AI-generated answer.

5?? JavaScript Functionalities

• File Upload Function (uploadFile())
• Question Submission Function (askQuestion())

6?? Responsive & Modern Design

• The layout adjusts dynamically for different screen sizes.
• Uses Bootstrap classes for:Containers (.container mt-5) → Centers content.Primary Buttons (.btn-primary) → Upload button. Success Buttons (.btn-success) → Ask question button.Text Styling (.fw-bold) → Highlights the chatbot’s response.

?? Step 3: Run the Flask App

Start the Flask server:

python app.py        

Then, open https://127.0.0.1:5001 in your browser.

?? Summary of Features

? Simple UI → Easy document upload & question input

? Local RAG System → Works offline using Mistral/Llama3

? Vector Database → Uses ChromaDB for fast retrieval

? Flask API → Lightweight & fast

? Bootstrap Styling → Clean, mobile-friendly UI

Now you have a fully functional Local RAG Chatbot with a Web UI! ????