The LOR Stack

During the late 1990s, the tech world adopted the LAMP stack, which consisted of Linux, Apache, MySQL, and PHP/Perl/Python. This combination was a game-changer for web development, clarifying the path through the internet's early and chaotic days.

Fast forward to a chat, I had about my GenAI for DevOps workshops. We ended up drawing parallels to the classic LAMP Stack, but for the GenAI era, we jokingly dubbed it the LOR Stack: Langchain/LlamaIndex, Observability, and RAG. It's a fresh take on today's challenges, where Langchain and LlamaIndex loosely play Apache's role by managing the operations of Large Language Models (LLMs). Rags introduces us to the Vector Database Management Systems (VDMS) concept, which is different but analogous to MySQL in that the data is not relational but in vector embeddings.?

Observability, or the "O" in LOR, shifts gears from traditional service monitoring to a nuanced view of LLM operations. It checks for relevance and correctness and ensures our GenAI isn't hallucinating. In one of my workshops, I showcased this process in action. A demo Python program in a Jupyter Notebook analyzed an academic paper, loaded it into MongoDB Atlas Vector Search, and then organized the data with Langchain's chunking. The open-source project Phoenix Arize monitored the accuracy of our queries and responses.

Let’s look at some of the code examples in the demo program.

This demo program loads a GPT-4 Technical Report research paper into a vector database. Langchain is used for data formatting (chunking), loading (embedding), and querying an LLM that uses the paper as the vector database,?

The first section of code installs the necessary Langchain tools, Arize's observability tool, and Pymongo (MongoDB Atlas Vector Search.)

We call the pdfloader library to get the research paper. After loading the PDF, we begin chunking. Chunking ensures that the data you are vectorizing is clustered correctly in the vector database and that retrial strategies are optimal.?

RecursiveCharacterTextSplitter is a common chunking strategy used by Langchain. This method does not split sentences, paragraphs, and pages in half.?

The next step in this Python program is to set up the vector database. In this example, we use Langhcain to convert our documents into vector embeddings by calling OpenAI’s default GPT foundational model. Langchain also calls the Pymongo API to load the embeddings into MongoDB’s Atlas Vector Database.

Here’s an example of how the data looks in MongoDB’s Atlas. Notice the field called “embedding,” which is the vectorized data. This field lists floating-point numbers in a highly dimensional vector representing the data's location. In this example, we used the OpenAI default embedding model, a 1536-dimensional vector. One of MongoDB's powerful features is combining JSON Document data with vector data in the same database and object.?

Once the PDF file has been converted into a vector database, we must tell Langchain to start a tracing session if we want to use the Phoenix Arize observability tool.? Following is an example of Langchain starting tracing. Phoenix Arize is open-source and straightforward to set up, which is one of my favorite things about it. They use “LLM as a Judge”? as their default evaluation setup. This allows you to skip taking a Coursera class to set up evaluation templates, flow, and ground truth. You can still use the deep dive version of evaluations with Arize if you want; you don’t have to.?

Now, we can run a query against the research paper PDF using Langchain and MongoDB’s Atlas Vector Search.?

Finally, we can look at the dynamic link Phoenix Arize created to see the trace. In future posts, I will demonstrate how to use evaluation tools (i.e., LLM Observability tools) to monitor correctness, relevance, and hallucinations.

The transition from LAMP to LOR represents our technological development as we embrace the intricacies and potential of GenAI. It demonstrates our ability to adapt and progress, discovering novel ways to innovate and propel ourselves forward. Digital narratives are constantly evolving.