Enriching LLms with context - vector - knowledge graph - causal

Background

Last year, I spent a lot of time thinking of the directions AI is going. This applies to our students at #universityofoxford and also our ?AI summit at Oxford

In a nutshell,? there are three directions we are tracking

1. AI augmentation?
2. AI reasoning and grounding
3. AI automation

In this newsletter, I will discuss AI grounding and why we emphasise knowledge graphs and causal graphs.?

Re-framing the problem as providing types of context for LLMs

To understand this better, we can reframe the problem as: “What are the types of contexts we could provide the LLM?”

Vector databases provide context in terms of proximity based search Hence, LLMs utilising vector databases are good at answering questions based on semantic similarity and contextual relevance,using embeddings.?

knowledge graphs provide context in terms of a subraph. LLMs integrated with knowledge graphs are better suited for answering questions that depend on explicit relationships and hierarchies between entities. They can leverage structured data to provide answers that require understanding of specific connections, such as familial relationships, organizational structures, or sequences of events within a defined context.

In this context, you can think of casual graphs as a special case of knowledge graphs. While Knowledge graphs capture a broad range of relationships (like associations, hierarchies, and affiliations) between entities, causal graphs focus on cause-and-effect dynamics, providing a structured way to model and infer the impact of one variable on another within a system.

Examples of usage

We can understand this better by considering examples:

content recommendation systems like Netflix could use vector databases to provide recommendations based on proximity searches even if the user hasn't explicitly searched for those topics.

To determine adverse reactions to drugs, a pharmaceutical company could use knowledge graphs to find possible risks based on defined relationships between entities.? providing precise and structured answers about compatibility or adverse reactions.

A healthcare company could use an LLM based on a causal graph to answer questions based on cause and effect relationships.?

In all cases, the context follows a familiar flow

1. Determine the context
2. Integrate the context with the LLM
3. Use the enriched LLM to provide responses to queries

Every form of context provides unique capabilities. For example, knowledge graphs provide explainability through traceable data sources, specific context provided by the developer, domain knowledge completeness of responses. We are working with neo4j in the integration of knowledge graphs and LLMs. In more complex cases, this integration leads to accelerated reasoning and decision making on the lines of the OODA loop (observe, orient, decide, act)

To recap, there are three directions we are tracking in the evolution of LLMs

• AI augmentation?
• AI reasoning and grounding and
• AI automation

I will explain the other trends in future posts?

You can meet us at the AI summit at Oxford

See our low code generative AI course for non developers at the #universityofoxford?

See our digital twins course at the University of Oxford?

Image source theaiedge.io

As seen at Erica Brown ’s post

UPDATE

I saw this post from Lulit Tesfaye - I think it captures well where we are going in terms of LLMs and context. LLMs leading to AGI will focus on general purpose models - but businesses will take those models and apply specific context to these models https://enterprise-knowledge.com/what-is-a-semantic-layer-components-and-enterprise-applications/