Using Patterns for Worry Free LLM Adoption
LLM and “Agentic” architectures are generating allot of excitement and showing potential for amazing efficiency gains in the enterprise that companies cannot ignore. But the technical stack used by LLMs and the immaturity of the frameworks, many less than a year old, gives valid reason to hesitate. We will how using tools common in the enterprise- tools like messaging systems, ESBs, ETLs- can be used to build more robust, performant, secure and reliable LLM applications, and without the need to adopt fragile frameworks or make exceptions to their existing policies.
In this article we separate the signal from the noise and demonstrate how use what is valuable, the LLM itself, while leaving behind hyped up tools and new fangled methodologies.
After analyzing a common LLM Framework called Retrieval Augmented Generation, or RAG for short, We will build the RAG framework using well known and trusted languages and tools that have proven their ability to be secure, scalable, reliable, and easy to use.
The advantage to this approach includes:
To adopt LLM in the enterprise we need to move past is aura of glossy newness surrounding LLM, role up or sleeves, and look into the library of SE best practices and review the list of familiar enterprise tools and technologies. To begin this journey let’s unpack the most familiar patter in the LLM world, the RAG. As we unpack it we will see that LLM is not so very different from what we are used to. Using patterns we gain mastery over LLM can build our own ad-hoc frameworks to the specifications of our unique use cases.
Best of all adopting LLM does not require us to forgo tools and computer language we trust.
RAG With Familiar Patterns
The most basic application using LLM is called Retrieval Augmented Generation or RAG for short. RAG is the most fundamental building block of LLM applications and, once understood, provides the background for understanding more complex agent based architectures.
Finding patterns in RAG
RAG is used to enrich prompts to the LLM with data that can be used by the LLM to augment its own knowledge store. Often a RAG is used when the user wants data that is stored in proprietary databases. For instance an auto part store my want to allow semantic search of its inventory. Likely the LLM no knowledge of the Auto parts store inventory in its database. The definition leads us to investigate the pattern Content Enricher (Figure 1)
Content Enricher - enriching the LLM prompt with proprietary data
The content enricher will access resources, such the inventory database in the case of the auto parts store. The extra data is added to the original message in a process called enrichment. The enriched data a now be forwarded to the LLM as part of the promo.
This will start to look familiar database engineer and many companies have significant investments in Messaging, ETL and ESB tools that support enrichment. There are also a number of open source tools available.
领英推荐
Smart Proxy - respond to each LLM client asyncronously
LLMs are non-deterministic and response times are dependent on the prompts, enrichment etc. In a busy system that might be handling many concurrent requests. In these kinds of scenarios using request-response can result in socket timeouts. To handle this we use a Smart Proxy pattern. (Figure 2)
A “smart proxy” is an asynchrony pattern that supports multiple requesters. The process LLM Proxy holds the identity of the requester while the LLM is processing the message. When the LLM finished processing proxy posts the response on the reply queue of the sender. The sender is is notified when the response is returned and can match requests and responses using a message id.
Channel Adapter - Making your LLM a first class citizen
LLM frameworks are very flexible and can be called by users through a Web UI or by a back office system. For instance the website may use LLM to summarize each product for the end user an also be used to extract arrival time from an api call with a shipping provider. The adapter pattern makes your LLM a first class citizen in the enterprise by providing interface for other back office components to integrate with the LLM framework as they would to other non-LLM components.
SUMMARY
LLM applications are just manifestations collections of design patterns. These design patters common in our IDEs, libraries and applications. Content enrichment is a common task for ESBs, and advanced messaging systems like Apache Synapse, Kafka, RabbitMQ and many others. Smart Proxy and Channel adapter are found in many tools and applications. In the Java world Spring Integration and Apache Camel both support all the patterns.
In the next article we will build RAG in the Java language using Spring Integration and RabbitMQ. I could as easily have picked any other language, NodeJS and Typescript would also be an excellent choice as would abiding applications with RUST, C++, RUBY, and any of the top 10 languages.