Unlock Data Insights with Semantic Labeling

Many businesses today are looking for ways to analyze user/human generated content on their platforms. Are conversations trending positively? Are there specific issues or topics occurring more often than not?

This 'natural language' content such as surveys, performance management notes, social media content run a into complicated issue - human made data

"How do I label human-content when the information they provide is inconsistent, poorly formed, or has jargon?"

Imagine your job is to read a piece of information, and label it multiple times depending on the subject, sentiment, outcomes, actions and more.

Here's an example: "I love your institution, everyone is always so nice to me. I wish there were better accommodations oh and can someone plz email me my latest account balance? thx!"

The above sentence is full of grammatical errors, short-hand, and multiple ways to measure sentiment.

Is this all Positive? Neutral-Positive? Hard to tell and also extremely subjective.

What you would normally do:

Traditionally, you would build a ML model by...

• Identify a large amount of historical, pre-labeled data
• Massage that data into a training set and test set
• Train a model and test for accuracy (and retrain if accuracy does not meet expectations).

This is a time-consuming process, riddled with manual effort, and can take weeks/months to implement depending on the cleanliness of the data infront of you.

Instead, let's try a novel approach to labeling... Semantic Labeling!

Using the power of a Vector database ( Weaviate ) and Anthropic on Amazon Web Services (AWS) Bedrock, we were able to quickly deploy and solve for this use case with minimal training effort.

The core concept is this...

Can we use the semantic meaning of synthetically generated user content, to identify the highest likelihood label to be assigned to a new piece of user content?

Here's how it works.

The goal is to break apart complex human conversations into...

• Manageable sentences
• Thematically or topically grouped content
• Using the same words and tone the user originally provided to maintain the authenticity of the request

First, let's prime the pump.

1. Take historical pieces of content (maybe a user comment in this scenario)
2. Use an LLM to break a part the content into thematically grouped sentences
3. Using these new sentences as samples -> generate semantically similar synthetic data
4. Store the synthetic data, pre-labeled in Weaviate

Now let's compare a new piece of content and find a label

Repeat this process for a new piece of content

1. Retrieve your new user content
2. Separate the thematically similar sentences via the LLM prompt
3. Do not generate synthetic variations

Perform a Semantic Label search!

Now with your new sentences, perform a looping semantic search over Weaviate .

• For Each sentence...
• Find the highest accuracy item in Weaviate based on your query
• Retrieve the label from the associated synthetic sentence from Weaviate
• Assign it to your new sentence and the original piece of content!

Pros and Cons of Semantic Labeling

When it comes to anything created by humans, the information can be subjective. People use different phrases and tone (like sarcasm) to create double-meaning. As a result you may need to...

• Cons: You will have to review your historic data for abnormal data points that don't fit a consistent theme or tone (an LLM can help identify these!)
• Cons: The prompt you write to generate sentences may need to be dynamic depending on where the content your analyzing originates from, if the users who create that content have their own community-language and lingo etc.
• Pros: You can easily human-in-the-loop content labeled in Weaviate, by validating the output label and confirming if the semantic meaning meets label expectations.
• Pros: Easily modifiable, is not expensive to train or implement
• Pros: COST EFFECTIVE, embedding and searching content in Weaviate can be an optimal use of dollars without long running inference requirements.

You can use Vector databases in unique ways

Hopefully this article show cases a unique way to leverage vector databases more importantly show cases the different ways to implement a solution such as Weaviate on Amazon Web Services (AWS) .

Not every Vector DB article needs to be about Chatbot RAG :)

If you're interested in solutions like this

Reach out to me and Innovative Solutions !

Our GenAI System "Tailwinds" solves for problems like these and many others.