Semantic Search Your Documents

Introduction

Semantic search is not a new thing, it has been there for a while that instead of searching specific keywords, we can query with meaning. that alone should be intriguing enough for most of us.

but even that can be done using any call to an LLM and providing a large text within the prompt itself. however, prompt size is still a finite number of characters and if we wanted to discuss a full PDF worth of content, this approach will not work.

what’s this Langchain thing.

so, a different approach is available and that includes using the most famous development framework for now at least, Langchain.

Langchain is a software development framework designed to simplify the creation of applications using large language models (LLMs) such as from OpenAI, Anthropic, or Hugging Face. The framework offers a suite of tools, components, and interfaces to manage interactions with language models, chain together multiple components, and integrate resources such as APIs, databases, and a wide variety of document types.

there 7 main components provided currently with Langchain and these are:

1. Schema: such as “text, documents, list of examples”
2. Models: such as OpenAI
3. Prompts: a component used to formulate the input into the structured template decided and use it as in input to the selected LLM
4. Indexes: basically a way to structure documents in order for LLM to interact with them.
5. Memory: to give our interaction with the LLM some context of the conversation.
6. Chains: one of my favorites, think of this as chain of components (LLM, Prompt, Index) in order to accomplish a use case.
7. Agents: a component that can analyze the users input, decide which tools to use and execute tasks by calling these tools.

and each one could have an article by itself.

Use-Case

I have built my own version of “chat with your document” project.

many people have already built something similar, mostly using some using Langchain Python, some used Langchain JS “as did i”.

Loading then Chatting

First, we need to load the document into a “vectorstore” DB in order for us to be able to query and chat with it later:

in case you’re wondering what is a vectorstore, the following is the description provided in the Langchain documentation:

A vector store is a particular type of database optimized for storing documents and their embeddings, and then fetching of the most relevant documents for a particular query, ie. those whose embeddings are most similar to the embedding of the query.

back to loading document step details:

• we need a document - I am using the “AI-Index-Report_2023”
• loading the PDF file into a DB.
• split the document into chunks.
• create embeddings.
• each embedding will use one of the text chunks to create a vector
• create a Vectorstore to store the vectors created by the embeddings.

once the loading (ingestion) is done, we can proceed with the Semantic search functionality (chatting with the document) which is using Langchain Chains:

To put the above image in words:

• we need a standalone question.
• embedded it to the vectorstore, making sure we are using the same Index we used for loading the document.
• vectorstore finds the most relevant information and pass it to the LLM
• LLM "GPT-3.5 in this case" formats the standalone question along with the result coming from the vectorstore and provides the final answer.

Code Overview

I won’t get into details with the code itself here but you can message me if you want more details, I’ll be happy to share them.

I wrote one method called queryPineconeStore(Question) and i will spit it into sections below for you to go through the code:

initialize the pinecone client and pass the pinecone API key to get the client ready to be used.

// Create Pinecone client and index

? ? const client = new PineconeClient();

? ? await client.init({

? ? ? apiKey: process.env.PINECONE_API_KEY,

? ? ? environment: process.env.PINECONE_ENVIRONMENT,

? ? });.        

embedding’s are essentials to either ingest the document text into vector store or even to retrieve it as we are doing here.

the index value should match the index you created when you loaded the document.

const embeddings = new OpenAIEmbeddings()

? ? const pineconeStore = await PineconeStore.fromExistingIndex(embeddings, {

? ? ? pineconeIndex: use the same index name when you loaded the document,

? ? });

? ? console.log("Pinecone store created successfully");;        

initialize the LLM, here using OpenAI

temperature : 0 to stop it from using its creativity while answering.

const model = new OpenAI({ temperature: 0 });        

create a “RetrievalQAChain” Chain and pass the model and the vector store we initialized above.

const chain = RetrievalQAChain.fromLLM(model, pineconeStore.asRetriever());        

call the chain by passing the question.

question here is basically an input taken from the user through a text field on the web page.

const response = await chain.call(

? ? ? query: question,

? ? });

? console.log({ response });

? ? return response;{        

a simple form for the user to ask a question.

submit and go through the process of retrieval explained above to get the result reutrned back on the same form

below is the section of the PDF representing the result:

obviously, this is not the best example to show, but it should at least give you an idea and you can definilty play with langchain to find the best way to implement this to your needs.

Potential Use Cases

• Medical search
• Legal search
• Corporate Content Search
• Calls and meetings transcripts search.

Conclusion

I am not sure if this article is doing #langchain the favor its due but if you are a developer or a tech company then you would do yourself a great service by looking into this framework.

check all its capabilities and the use cases they already implemented as they just included #babyagi and #autogpt into their framework as well.

if you plan on getting involved in any AI development in the future for personal or business purposes, then i believe this should be you immediate next step.

References

A. Langchain Documentation (JS version): Welcome to LangChain | ????? Langchain

B. YouTube Channel: https://www.youtube.com/@samwitteveenai who inspired this article, I used some of his images as well.

C. Sam's Python Collab URL for a better python version of the same functionality: https://colab.research.google.com/drive/13FpBqmhYa5Ex4smVhivfEhk2k4S5skwG?usp=sharing