Leading Java AI Frameworks: LangChain4j vs Spring AI for Custom Chatbots ??

In our previous articles, we discovered the core concepts of LLMs and RAG, also exploring the RAG process under the hood. During this article, we're diving into the exciting world of AI frameworks for Java. If you've been scratching your head trying to figure out which framework to use for your next AI project, you're in the right place. We're going to break down the differences between Langchain4j and SpringAI, and highlighting their unique features and capabilities.

Quick Reminder: Language Models, Chat Models, Embedding Models, and RAG

Before we jump into the nitty-gritty of each framework, let's revisit some key concepts that we've covered in our previous articles:

• Language Models are AI models trained on big amounts of text data to predict the next word in a sequence that they can then generate a coherent text based on the provided input.
• Chat Models: These are fine-tuned language models designed to facilitate conversations, answer questions, and respond in a conversational manner, they can maintain context over multiple turns and give coherent responses. They are typically used for interactive applications like chatbots.
• Embedding Models: These models are used to convert text into numerical vectors (embeddings) to represent the text's semantic meaning. They are important in RAG because they allow us to search for relevant information in the text based on similarity search algorithms (Go back to RAG Under the Hood ?? if you need a quick refresher on this topic).

• RAG: It's a technique that combines the generative capabilities of LLMs with the advanced search capabilities of vector databases from an external knowledge base. This approach enhances the accuracy of the responses and gives up-to-date and specific responses. (Go back to RAG Under the Hood ?? if you need more details about how RAG process works).

Now that we're all on the same page, let's dive into our contenders!

The Contenders in the Ring ??

LangChain4j ?????: Components and Features

LangChain4j is a Java framework designed to simplify the development of LLM/RAG applications in Java ecosystem based on LangChain. It provides a set of abstractions and tools to work with LLM providers like OpenAI, Ollama, and more and also allows implementing RAG workflows for specific use cases.

Key Components

• Model interfaces: LangChain4j provides a set of interfaces to work with different types of models, like: LanguageModel, ChatLanguageModel, EmbeddingModel, and TokenizerModel. These interfaces allow seamless integration with various AI providers such as OpenAI, Ollama, Hugging Face, and many others.
• Document Loaders: As we've previously seen, the RAG process requires a lot of data, and in most cases, this data is stored in a vector database. LangChain4j provides components facilitating loading documents from different sources (e.g., PDF, text files, etc.) into a common format.
• Text Splitters: These are used to split the text into smaller chunks for efficient processing and embedding. LangChain4j provides a set of text splitters to work with different types of text, like: RecursiveCharacterTextSplitter, TokenTextSplitter, and SentenceTextSplitter. These allow you to split text based on character count, token count, or sentence boundaries.
• Vector Databases: Abstractions for storing and retrieving vector embeddings, with support for various vector databases. The most popular choices are PgVector and Chroma.
• Memory: It allows for maintaining a conversation history and context, which is crucial for chatbot applications.
• Agents: Tools for composing multiple steps in an LLM workflow.

There are a lot more components that LangChain4j provides, and you can find more details in the official documentation.

Example: Implementing RAG with LangChain4j

Here's a simple example of how to implement RAG with LangChain4j.

In this sample example, we've walked through all the necessary steps to implement RAG with LangChain4j, from loading documents and generating embeddings to creating the final prompt and generating answers.

For more advanced use cases, you can use the appropriate abstractions, like the Vectore database to store embeddings also choose the appropriate Text Splitter in case you need to split the text in a specific way.

To see LangChain4j in action, check out a real-world example I built: a Spring Boot documentation chatbot. This project demonstrates how to create a chatbot that quickly and accurately answers questions about Spring Boot documentation using RAG techniques. GitHub Repo ??

Spring AI ??: Components and Features

Spring AI is a sub-project of Spring developed by the Spring team to integrate AI capabilities into Spring applications. It aims to provide components and features for working with LLM providers and RAG workflows like Langchain4j project.

Key Components

• AI Client: Interface to for interacting with AI services, supporting both synchronous and asynchronous operations.
• Model Abstractions: Same as LangChain4j, Spring AI provides a set of abstractions to work with different types of models (e.g., OpenAI, Ollama, etc.)
• Prompt Template: Spring AI includes a PromptTemplate class that helps in creating and managing prompts with placeholders for dynamic content.
• Document Reader: Spring AI gives us the DocumentReader interface and an HtmlDocumentReader implementation to read and process documents from various sources.
• Transformers: Utilities for processing and transforming text data like tokenization, sentence splitting, and more.
• Vector Database

You can find more details about each component in the official documentation.

Example: Implementing RAG with Spring AI

This code demonstrates how to configure and utilize Spring AI to implement a RAG system. We've created a simple RAGService that uses Spring AI components to perform three key functions: indexing documents, retrieving relevant information, and generating insightful answers.

To see Spring AI in action beyond this basic example, check out another real-world project I developed: the Moroccan Cooking Companion ???????. This Spring Boot application allow users to explore traditional Moroccan cuisine by searching for dishes and retrieving not only their recipes but also appetizing images. The project built with Spring Boot, Spring AI, and Hilla, demonstrating how these technologies can come together to create a rich, interactive experience. GitHub Repo ??

Comparison: Langchain4j vs SpringAI

To provide a comparison of Langchain4j vs SpringAI, let's examine a comprehensive overview of these frameworks along with another player in the field, Semantic Kernel. This comparison is based on data collected in February 2024.

Let's summarize the key points from this comparison:

Wrapping It Up

Both Langchain4j and SpringAI offer powerful tools for building RAG applications in the Java ecosystem. The choice between them will depend on your specific needs, existing technology stack, and development preferences.

Langchain4j provides a rich set of components and higher-level abstractions, making it an excellent choice for developers looking to quickly build complex AI applications. Its wide range of integrations and extensive documentation make it a versatile option for various use cases.

SpringAI, on the other hand, shines in its seamless integration with the Spring ecosystem. It's an ideal choice for teams already using Spring and looking to add AI capabilities to their applications. While it may have fewer features compared to Langchain4j, its design and the backing of the Spring team make it a strong framework, especially for enterprise applications.

As both frameworks continue to evolve, we can expect to see more features, optimizations, and integrations added over time.