Empowering ChatGPT with Scalable Vector Search: A Qdrant on AKS Case Study

Introduction

Large language models (LLMs) like ChatGPT are transforming communication and information access. However, as these models grow in capability, so do the challenges in efficiently retrieving information. In this post, we explore how we enhanced a client’s ChatGPT application by implementing a scalable vector search solution using Qdrant on Azure Kubernetes Service (AKS).

?

The Challenge: Scaling ChatGPT with File Upload Search

Our client’s ChatGPT application was performing well; however, we saw opportunities for significant enhancements. One major feature addition was the ability to search uploaded files, which required a robust vector search solution. Here were the key challenges we faced:

• Efficient Information Retrieval: Traditional keyword-based search struggles with unstructured data like uploaded files. Vector search excels at identifying semantic similarities, making it perfect for searching through these files.
• Scalability and Redundancy: As user adoption and file uploads grow, the search system must scale efficiently and maintain redundancy to ensure continuous service.
• Secure Communication: Ensuring secure communication between the ChatGPT application and the vector search solution is crucial, especially in a production environment.

?

Qdrant - A High-Performance Vector Database

To address the above challenges, we chose Qdrant, an open-source vector database designed for high-performance vector similarity search. Here’s why Qdrant stood out:

• Efficient Vector Storage and Retrieval: Qdrant uses HNSW (Hierarchical Navigable Small World) graphs and faiss (Facebook AI Similarity Search) libraries for efficient handling of high-dimensional vectors.
• Scalability and High Availability: Qdrant supports horizontal scaling, allowing us to add nodes as data volume and query load increase. It also offers replication for high availability, ensuring service continuity during node failures.
• Flexibility with Embedding Techniques: Qdrant is agnostic to embedding techniques, supporting popular libraries like Sentence Transformers or Gensim.
• Seamless Integration: Qdrant provides client libraries for various programming languages, including Python, making integration with existing application stacks straightforward.

?

Building a Scalable Qdrant Cluster on AKS

To achieve scalable vector search with Qdrant, we opted for a managed Kubernetes approach using Azure Kubernetes Service (AKS) from Microsoft. This eliminated the resource burden of deploying and managing a standalone cluster. AKS orchestrates a highly available Qdrant cluster with three nodes for redundancy and future scaling. Secure communication between the application servers and the Qdrant cluster is ensured through an internal load balancer within the AKS environment. Here is how we did it:

• Provisioning a 3-Node AKS Cluster: We created a highly available AKS cluster with three nodes to ensure redundancy and scalability for Qdrant.
• Streamlined Deployment with Helm Charts: We used a Qdrant Helm chart to simplify the deployment process within the AKS environment.
• Internal Load Balancer for Secure Communication: We configured an internal load balancer for the Qdrant cluster to ensure that only authorized application servers could access the Qdrant service.
• Virtual Network (VNet) Peering for Seamless Interaction: We implemented VNet peering to allow secure communication between the ChatGPT application (residing in a separate VNet) and the Qdrant cluster.

?

Implementation Details and Considerations

Here’s a deeper dive into some critical implementation aspects:

• Data Preprocessing and Embedding: Before indexing data in Qdrant, we used text preprocessing techniques like tokenization, stop word removal, and stemming. We then used a pre-trained sentence embedding model (e.g., Sentence Transformers) to generate dense vector representations of the textual content within uploaded files, which were indexed in Qdrant.
• Fine-tuning the Search Experience: Qdrant offers various parameters for fine-tuning the search experience. We experimented with different distance metrics (e.g., cosine similarity) and filtering options based on metadata to optimize search results.
• Monitoring and Logging: For proactive management, we used Azure Monitor along with AKS. Azure Container Logs (ACL) were used to collect logs from the Qdrant cluster, and Prometheus was used for monitoring.

?

This blog post explores how we enhanced a client's ChatGPT application with scalable vector search using Qdrant on Azure Kubernetes Service (AKS). Traditional search methods struggled with the client's need to search uploaded files. Qdrant, an open-source vector database, addressed this with its efficient vector storage and retrieval along with scaling capabilities. AKS, a managed Kubernetes offering, simplified deployment and management of the Qdrant cluster. We ensured secure communication through an internal load balancer and VNet peering. This approach provides a robust, scalable, and secure solution for searching uploaded files in large language models like ChatGPT.