Unknown Unknowns: Why AI Needs Knowledge Graphs

Scenario

Logan is an experienced mechanic. He is in the first few months of a new job as a maintenance worker in a food production company. The company has just deployed a fancy new AI RAG search tool that promises to provide easy access to manuals and procedures. Logan thinks: “The demo was amazing. I could certainly use this AI thing to help me learn all the new procedures in this job. I just need to talk to it and it will answer all my questions. Fantastic!”

He gets started on his shift and asks his first question: “Belt cleaning”. The answer is pretty good. It doesn’t hallucinate an answer; instead, it provides a detailed procedure referencing “standard procedure for cleaning a conveyor belt” on page 123 of a PDF on SharePoint. Feeling good, he completes the process, just as instructed by the AI.

----

The next day, his supervisor Santiago pulls him aside:

Santiago: “We’ve got an outage on belt #15, what the hell did you do to it?!”

Logan: “I just cleaned it, like the AI told me to.”

Santiago: “Don’t you know that the location has high humidity and therefore requires dry steam cleaning? And didn’t you read the notice from the manufacturer that advises that all Novacore F6 belt rollers should be re-attached with 7 Nm of torque after cleaning? Now products are slipping off the wet belt and the belt is drifting because of misaligned rollers.”

Logan: “Um, well, the AI didn’t tell me any of those things. How was I supposed to know?”

?

What happened here?

Logan was using a RAG (Retrieval Augmented Generation) search system that finds documents semantically related to a user question, and then uses a LLM (Large Language Model) to answer that question with the related documents' text as additional context. These systems are great for answering questions with less hallucinations.

The problem is the unknown unknowns, or false negatives, of such answers. I.e. what did semantic search not find that would have been relevant to the answer? How does the user know that all relevant information was included when answering the question? In the scenario above, the cleaning work was to be performed on a conveyor belt, that belt is in a specific location, and that location had an environmental hazard that Logan should have taken into account. The belt also had a specific make and model with a notice from the manufacturer.

Logan’s question of “belt cleaning” did not pull in all the relevant information. The information might have been there somewhere in the company’s systems, but, because the semantic search couldn’t find it, it was impossible for the LLM to correctly answer the question.

A better query might have been: “What is the company best practice for cleaning the Fenner F6 Novacore conveyor belt rollers in the western-side location, considering any environmental hazards flagged for that location, and also look up company notices for that model of belt?” But no one is going to have the foresight to write such a detailed prompt.

Why are Knowledge Graphs Important?

A knowledge graph is a data structure that links together nodes of data. We can structure all the data in a business into a graph, and then query that graph for a specific answer. Querying involves an AI agent walking through the graph, jumping from node to node, gathering insights along the way. The graph queries gives the agent all the specific relevant context it needs to answer a user’s questions without hallucinating (false positives) and without missing any important details (false negatives).

Building such a graph-backed LLM system from scratch is a non-trivial project. It requires teams to:

1. Create an ontology with all the relevant concepts in the business domain.
2. Build a knowledge graph that links together and transforms existing data from multiple different systems of record.
3. Teach an AI agent how to walk the graph to get relevant information to answer a user's question.
4. Turn the resulting sub-graph into a useful answer for the user.
5. Evaluate and continuously optimize the whole process.

At Datch, we offer products and services for every step above, and we love working with customers to create effective solutions. Please get in touch if you would like to know more.