LLMs and chatbots: a brief update

Written by Chris King

Generally and historically, data engineering, analytics, and science efforts focused on progressing from data to knowledge/wisdom. The emergence of LLMs allows for the decomposition of wisdom/knowledge back down to data. This can enable novel discovery, integrate with information systems, and drive automated processes.

GenAI Categories

• Generation: Use bedrock models to create code, text, or images
• Summarization: Extracting concise, key details from larger bodies of text
• Entity and Relationship Extraction: Identify relevant nouns, verbs, and interactions from unstructured text
• ChatBots: Chat interfaces for predefined workflows

LLM’s & Graph Databases

• Graph databases excel at representing entities and relationships using nodes and edges
• LLMs can be primed and prompted to extract these entities and relationships and write the output to a graph database
• Graph databases can represent this data via knowledge graphs

Knowledge graphs can:

• Represent entity/relationships visually for serendipitous discovery
• Be queried via OpenCypher/CQL
• Interacted with programmatically via API’s

For example, the following paragraph is from a Wiki article on Galileo, and the image shows the nodes and edges extracted by an LLM and stored in Neo4j.

“Galileo studied speed and velocity, gravity and free fall, the principle of relativity, inertia, projectile motion and also worked in applied science and technology, describing the properties of pendulums and “hydrostatic balances”. He invented the thermoscope and various military compasses, and used the telescope for scientific observations of celestial objects. His contributions to observational astronomy include telescopic confirmation of the phases of Venus, observation of the four largest satellites of Jupiter, observation of Saturn’s rings, and analysis of lunar craters and sunspots.”

Entity Extraction

Knowledge Graphs with LLMs

1. Parse the unstructured text into chunks
2. Extract the entities and relationships from the summarized text
3. Write the entities and relationships to the database
4. Provide the db schema to an app to answer the given question
5. Query the database for relevant nodes
6. Respond with a well-formatted answer

Legal Text Parsing

Another example, this case is a legal document describing maximum rates for energy transmission and the corresponding image shows not only nodes and edges but also the properties required to calculate cost.

For Point-to-Point service reserved for an Annual Period or a Monthly Period, the charge for service supplied in a Monthly Period shall not exceed the Transmission Customers Monthly Period transmission reservation multiplied by $36.50 per MW-month. For a Network Integration Transmission Service Customer, the charge for service supplied in a month shall be the Customers load coincident with the hour of the DEP monthly Transmission System Peak during the month, multiplied by $36.50 per MW.

LLMs and Knowledge Graphs together can unlock a wealth of previously accessible data<=>wisdom. Some example use cases:

• Obituary analysis for fraud prevention
• Tariff analysis for the energy sector
• Cyber Threat Intelligence report analysis for cyber security

Chatbots

Problem Statement: In a large or growing organization, it’s difficult to navigate policy and rules. Even if you know where all of the company policies live, knowing how they apply in various situations or geographies can be a challenge. You could ping your HR representative, and they are more than happy to help you, but what if you could ask a chatbot versed in your company’s specific guidelines?

Solution: Enter HR Bot, the LLM powered chatbot steeped in your company’s internal documentation. Now you can ask any question and get answers instantly, and without diverting resources away from critical administrative tasks!

Chatbots with Retrieval Augmented Generation (RAG)

• The user makes a request to the server with a question
• Lambda vectorizes the question
• Lambda fetches relevant docs using the Vector Store top 5 vectors
• Lambda prompts the LLM with the user’s question and related vectors
• Lambda sends the formatted response from Model back to Slack and the user

Conclusion

ChatBots allow you to quickly automate the vast majority of customer interactions while ensuring high quality, detailed information, and a friendly tone while gaining valuable insights from chatbot metrics to drive improvements. Predefined workflows provide business rules and establish guardrails for bot interactions. LLMs provide deeper insights into datasets and are useful in converting unstructured text into structured data stores.