Langchain4J using Redis

I've developed a proof-of-concept that leverages Langchain4J for semantic search in combination with Redis as the vector database. Below is a concise guide outlining how to set this up, complete with Java code examples and steps to configure a cloud-based Redis instance.

Redis setup

I've employed a free-tier cloud Redis instance (v6.2.6) that comes with the "Search and Query" modules pre-installed. Connection details such as the URL, port, and password are readily available, and the default user for the instance is named "default."

In under a minute, you'll have a fully operational Redis instance, ready for you to start tinkering with.

???? https://app.redislabs.com

Langchain4J setup

I'm using version 0.23.0 of Langchain4J, which you'll need to include in your Maven POM file or Gradle build script to get started.

        <dependency>
            <groupId>dev.langchain4j</groupId>
            <artifactId>langchain4j</artifactId>
            <version>0.23.0</version>
        </dependency>        

The Devoxx talks

For individual Devoxx and VoxxedDays CFP instances, I'm using an in-memory embedding store to search the data. However, there's also an aggregated instance where all the talks are consolidated. For this unified instance, I wanted to use a persistent vector database, so I opted to experiment with Redis.

This Devoxx Belgium talk by Alexander Chatzizacharias discusses other vector database options.

The Redis Embedding Store

Langchain4J comes with a built-in RedisEmbeddingStore that you can easily configure to suit your needs. Here's a sample configuration guide to get you started. By following these steps, you'll link Langchain4J with your Redis instance, ensuring that your embeddings are persistently stored.

//  Create Redis connection using custom metadata fields
var metadata = List.of(Converters.TALK_ID, Converters.TALK_TITLE);

redisEmbeddingStore = RedisEmbeddingStore.builder()
            .host(redisHost)
            .user("default")
            .password(redisPassword)
            .port(redisPort)
            .dimension(384)
            .metadataFieldsName(metadata)
            .build();        

The host, user, password, and port settings are straightforward, serving to establish the connection to the Redis instance. The vector size dimension for the Redis embedding is set to 384, which specifies the number of features in the embedding vector.

I did encounter some initial hiccups with the metadata fields not mapping correctly. After some digging, I discovered that these needed to be supplied as a list in the builder. I'm using this feature to keep track of extra meta fields like talk_id and talk_title. This enables the semantic search results to display these additional details alongside the similarity score or distance.

Embedding The Content

To embed your content, you'll need to choose a (mini) large language model. You're not limited to using Langchain4J's built-in models; you can also opt for others like OpenAI's GPT models or even custom-trained ones.

// Using the SentenceTransformers all-MiniLM-L6-v2 embedding model
EmbeddingModel embeddingModel = new AllMiniLmL6V2EmbeddingModel();        

To get your content, you'll first need to fetch the talks, which in my case was done via a REST API call.

Here's a skeleton Java method to convert a talk object into a TextSegment, which can then be fed into the language model for embedding:

/**
 * Convert talk to text segment
 * @param talk the talk to convert
 * @return the text segment
 */
public static TextSegment toSegment(Talk talk) {
  String presenterDetails = getPresenterDetails(talk);
  String description = removeHtmlElements(talk.description());
  String tags = concatenate(talk.tags());

  String text = String.format("'%s' by %s\n%s\n%s", talk.title().trim(), presenterDetails, description, tags);

  Metadata metadata = createMetadata(talk);

  return TextSegment.from(text, metadata);
}

/**
     * Create metadata for the talk
     * @param talk the talk
     * @return the metadata
     */
private static Metadata createMetadata(Talk talk) {
  return new Metadata()
                .add(TALK_ID, talk.id())
                .add(TALK_TITLE, talk.title());
}        

Once you have a way to convert talks into TextSegments, you're all set to work some vector embedding magic. You can stream through your list of talks, transform each one into a TextSegment, and then feed it to your chosen language model for embedding.

Here's a simple Java code snippet illustrating how you might do this:

// Convert content to text segments
List<TextSegment> segments = talks.stream()
                .map(Converters::toSegment)
                .toList();

// Create embedding vectors
Response<List<Embedding>> listResponse =  
                          embeddingModel.embedAll(segments);

// Store vectors and text into Redis
redisEmbeddingStore.addAll(listResponse.content(), segments);        

Awesome, you've successfully populated your Redis instance with vector embeddings along with their associated metadata. You're now all set for doing some semantic search.

A truncated extract from the Redis index would include the vectors and the metadata fields (talk_id and talk_title), making it both machine-readable for vector operations and human-readable for understanding the context.

"{
\"talk_id\":\"1\",
\"vector\":[0.0036637965,-0.02023412,0.12321208,-0.17431667,0.008200323],...],
\"talk_title\":\"Empathetic communication at work\",\"text\":\"'Empathetic communication at work' by Sharon Steed (Communilogue LLC)\\nCommunication is the one skill that will never be obsolete. Why, then, are so many of us intimidated by working on or beefing up our communication strategies? We can all be great communicators, we just need to learn to approach it from a place of empathy. This interactive workshop tackles your communications fears and weaknesses head on with a full day focused on both sides of the conversation spectrum: speaking and listening. The main goal of this workshop is to learn how to approach every interaction from a place of empathy. \\n\\n1. Understand the true purpose of listening\\n2. Learn to shift listening into an action-based activity \\n3. Identifying biases and learn how to combat them\\n4. Learn how to fight fair and use those disagreements as gateways to collaboration\\n5. Learn how to craft responses that best helps your listeners understand and embrace your message\\ncommunication, collaboration, Community networking, empathy\"}"        

The actual Semantic Search

With Redis now populated, we're ready to execute some semantic search queries.

// Embed the search query using AllMiniLmL6V2EmbeddingModel()
Response<Embedding> queryResponse = embeddingModel.embed(query);

// Find the relevant talks using Redis embedding store
List<EmbeddingMatch<TextSegment>> matches =
          redisEmbeddingStore.findRelevant(queryResponse.content());

// Return the results
return Optional.of(matches.stream()
                .map(Converters::toSearchResultItem)
                .toList());        

For example when I search on "emotions" I get the following result :

[
    {
        "score": 0.763107001781,
        "id": 1,
        "title": "Empathetic communication at work"
    }
]        

The Converter maps the found metadata fields to "id" and "title" and includes the score.

This is just too easy ??

-Stephan

BTW You might also want to check out this very interesting Langchain4J overview talk from Lize Raes from Devoxx Belgium 2023.