Get Started With GraphRAG

GraphRAG Overview

Retrieval-Augmented Generation (RAG) is a technique to search for information based on a user query and provide the results as reference for an AI answer to be generated. This technique is an important part of most LLM-based tools and the majority of RAG approaches use vector similarity as the search technique.

GraphRAG uses LLM-generated knowledge graphs to provide substantial improvements in question-and-answer performance when conducting document analysis of complex information.

With the rise of Generative AI the ability for an application to provide comprehensive and explainable results has become critical. While architectures such as Retrieval Augmented Generation (RAG) have become popular ways to help mitigate these issues, combining them with knowledge graphs, using a technique called GraphRAG, has proven to be a valuable tool for getting better results.

Vector and graph queries each add value in RAG.This is distinct from “vector-only RAG,” which is strictly based on similarity with embeddings based on words in text.

GraphRAG Pattern

Fundamentally, GraphRAG is RAG, where the Retrieval path includes a knowledge graph. As you can see below, the core GraphRAG pattern is straightforward. It’s basically the same architecture as RAG with vectors but with a knowledge graph layered into the picture

For us, it’s a set of RAG patterns that leverage a graph structure for retrieval. Each pattern demands a unique data structure, or graph pattern, to function effectively.

Here, you see a graph query being triggered. It can optionally include a vector similarity component. You can choose to store your graphs and vectors either separately in two distinct databases, or use a graph database like Amazon Neptune which also supports vector search.

One of the common patterns for using GraphRAG is as follows:

1. Do a vector or keyword search to find an initial set of nodes.
2. Traverse the graph to bring back information about related nodes.
3. Optionally, re-rank documents using a graph-based ranking algorithm such as PageRank.

Patterns vary by use case, and like everything else in AI today, GraphRAG is proving to be a rich space, with new discoveries emerging every week.

GraphRAG Lifecycle

A GenAI application that uses GraphRAG follows the same pattern as any RAG application, with an added “create graph” step at the start:

Creating a graph is analogous to chunking documents and loading them into a vector database. Advances in tooling have made graph creation literally that easy. The good news is threefold:

1. Graphs are highly iterative – you can start with a “minimum viable graph” and expand from there.
2. Once your data is in a knowledge graph, it becomes very easy to evolve. You can add more kinds of data, to reap the benefits of data network effects. You can also improve the quality of the data to up the value of your application results.
3. This part of the stack is rapidly improving, which means graph creation will only get easier as tooling gets more sophisticated.

Benefits of GraphRAG

The benefits we are seeing from GraphRAG relative to vector-only RAG fall into three main buckets:

1. Higher accuracy and more complete answers (runtime / production benefit)
2. Once you’ve created your knowledge graph, then it’s easier to both build and subsequently maintain your RAG application (development time benefit) i.e Improved Data Understanding, Faster Iteration
3. Better explainability, traceability, and access controls (governance benefit) i.e Governance: Explainability, Security, and More

Knowledge Graph

A knowledge graph model is especially suitable for representing both structured and unstructured data with connected elements. Unlike traditional databases, they do not require a rigid schema but are more flexible in the data model. The graph model allows efficient storage, management, querying, and processing of the richness of real-world information. In a RAG system, the knowledge graph serves as the flexible memory companion to the language skills of LLMs, such as summarization, translation, and extraction.

In a knowledge graph, facts and entities are represented as nodes with attributes connected with typed relationships, which also carry attributes for qualification. This graph model can scale from a simple family tree to the complete digital twin of a company encompassing employees, customers, processes, products, partnerships, and resources, with millions or billions of connections.

Graph structures can originate from various sources, from a structured business domain, (hierarchical) document representations, and signals computed by graph algorithms.

When we dive into retrieval patterns, we notice how the most advanced techniques rely on the connections within the data. Whether it’s metadata filtering, like searching for articles by a specific author or on a particular topic, or parent-child retrievers, which navigate back to the parent of a text chunk to provide breadth to the LLM for context-enriched answers, these methods leverage the relationships between the data to be retrieved.

Typically, these implementations rely heavily on client-side data structures and extensive Python code connecting the different pieces of information. However, in a graph database, establishing real relationships and querying them with simple patterns is much more efficient.

In the graph pattern of almost every pattern, you will see the entities or domain nodes that represent your application domain domain relationships document nodes that represent the unstructured documents ingested into the graph.

Natural language querying allows users to interact with computer systems using everyday human language, rather than having to learn and use structured query languages or complex programming commands. This capability enables users to ask questions or provide instructions in their native tongue, and the system can then process this input to understand the user's intent and provide relevant information or perform the requested action.

However, in many real-world applications, it is not safe or appropriate to allow users to ask completely open-ended questions of a data set. For example, in a banking application, you may want to give users the ability to ask about their own transactions and account balance, but you certainly wouldn't want to allow them to inquire about other users' financial information.

To address these types of use cases, we can leverage a technique called Knowledge Graph Retrieval. Here, a large language model (LLM) is used to extract key entities from the user's natural language question. These extracted entities are then used as the parameters for a pre-defined, templated query. This approach gives the application developer the freedom to create optimized, secure queries with appropriate data access controls, while still providing users with a natural language interface on top of the underlying data.

Chunking

Text documents can be short (a social sedia post or comment) or very long (a book).

As longer text documents cover many different topics in sequential order (sometimes with references), it is desireable to structure them into smaller pieces that are semantically coherent and focus on one topic.

This process of splitting up documents into smaller pieces is called Chunking.

There are a number of different chunking strategies, listed below.

• Splitting: Split a document into equal sized sections (by character or token-count), with an optional overlap (typical sizes are 250-500 tokens with 50-100 tokens overlap)
• Hierarchical Document Chunking: Split a document alongside lexical boundaries - chapters, sections, paragraphs
• Sentence Chunking: Split a document into individual sentences
• Semantic Chunking: Split documents into sentences, generate embeddings and split on boundaries where the embedding vector distance exceeds a certain threshold

Extracting Nodes and Relationships

The next step is constructing knowledge from text chunks. For this use case, we use an LLM to extract structured information in the form of nodes and relationships from the text. You can examine the LLM prompt the authors used in the paper. They have LLM prompts where we can predefine node labels if needed, but by default, that’s optional. Additionally, the extracted relationships in the original documentation don’t really have a type, only a description. I imagine the reason behind this choice is to allow the LLM to extract and retain richer and more nuanced information as relationships. But it’s difficult to have a clean knowledge graph with no relationship-type specifications (the descriptions could go into a property).

GraphRAG-Amazon Bedrock with Neptune

Amazon Bedrock Knowledge Bases offers fully-managed, end-to-end Retrieval-Augmented Generation (RAG) workflows to create highly accurate, low latency, and custom Generative AI applications by incorporating contextual information from your company's data sources. Amazon Bedrock Knowledge Bases now offers a fully-managed GraphRAG capability with Amazon Neptune Analytics.

Previously, customers faced challenges in conducting exhaustive, multi-step searches across disparate content. By identifying key entities across documents, GraphRAG delivers insights that leverage relationships within the data, enabling improved responses to end users. For example, users can ask a travel application for family-friendly beach destinations with direct flights and good seafood restaurants.

Developers building Generative AI applications can enable GraphRAG in just a few clicks by specifying their data sources and choosing Amazon Neptune Analytics as their vector store when creating a knowledge base. This will automatically generate and store vector embeddings in Amazon Neptune Analytics, along with a graph representation of entities and their relationships.

GraphRAG with Amazon Neptune is built right into Amazon Bedrock Knowledge Bases, offering an integrated experience with no additional setup or additional charges beyond the underlying services

Amazon Bedrock Knowledge Bases offers a fully managed GraphRAG feature with Amazon Neptune. This functionality uses Retrieval Augmented Generation (RAG) techniques combined with graphs to enhance generative AI applications so that end users can get more accurate and comprehensive responses.

GraphRAG automatically identifies and uses relationships between related entities and structural elements (such as section titles) across documents that are ingested into Amazon Bedrock Knowledge Bases. This means that generative AI applications can deliver more relevant responses in cases where connecting data and reasoning across multiple document chunks is needed.

Amazon Bedrock Knowledge Bases automatically manages the creation and maintenance of the graphs from Amazon Neptune, so you can provide relevant responses to your end users, without relying on expertise in graph techniques.

Amazon Bedrock Knowledge Bases with GraphRAG offers the following benefits:

• More relevant responses by using contextual information from related entities and document sections.
• The summarization by incorporating key content from your data sources while filtering out unnecessary information.
• More explainable responses by understanding the relationships between different entities in the dataset and providing citations.

Using graphs with Amazon Neptune just got a whole lot simpler with Langchain and LLMs! Customers use graphs with Amazon Neptune for use cases like detecting fraud, customer 360, and security posture assessment. They also use Amazon Neptune to build knowledge graphs that connect disparate data sources.

Implementing GraphRAG offers a powerful approach to using the strengths of both LLMs and structured knowledge bases. Integrating the natural language understanding capabilities of language models with the factual and contextual information contained within knowledge graphs, GraphRAG enables the generation of highly accurate and informative responses. Amazon Bedrock and Amazon Neptune to facilitate the implementation of GraphRAG and increase the capabilities of generative artificial intelligence (AI)-powered applications.

GraphRAG- databases

A graph database allows you to model, store and query your data in the form of a graph, or network structure. Graphs and graph databases are ideal for applications where you need to understand the connections between things, the nature or semantics of those connections, and the strength, weight or quality of each connection.

Which are purpose-built to store and navigate relationships - have emerged as a better-fit data store for identity graph solutions. Graph databases are easy to model for highly connected data, treat relationships as “first class citizens,” have flexible schemas, and provide higher performance for graph query traversals. Using a graph database for an identity graph enables you to link identifiers and update profiles with more easily and query at ultra-low latency — enabling faster updates and more accurate, up-to-date profile data for ad targeting, personalization, analytics, and ad attribution.

With a graph database you can identify patterns of connected data, follow paths that connect entities, identify indirect and transitive relationships that might not otherwise be apparent on the data, and analyze the communities to which entities belong, and the relative importance or influence of different entities within a connected structure.

Example graph database workloads include:

• Knowledge graphs We discussed knowledge graphs above. Today, knowledge graphs are being used more and more across industries and verticals to generate insights into complex business and technical domains.
• Fraud detection By connecting seemingly isolated facts graphs can help us find patterns of fraudulent behaviors.
• Identity graphs Used for real-time personalization and targeted advertising, these graphs connect devices, browsing data, and identity information to create a single unified view of customers and prospects based on their interactions with a product or website.
• Security graphs Graphs are ideal for modelling the connections between assets and roles within your organization. Security graphs can be used for proactive detection, reactive investigation, and as part of a defense-in-depth strategy for improving your IT security

Introducing Amazon Neptune Amazon Neptune is a fast, reliable, and fully managed graph database service for building and running applications with highly connected datasets, such as knowledge graphs, fraud graphs, identity graphs, and security graphs.

https://aws.amazon.com/neptune/

Amazon Neptune provides two different engine offerings, Neptune Database and Neptune Analytics. You can use with this notebook however some differences may apply for certain cells.

Neptune Database is a serverless graph database designed for superior scalability and availability. Neptune Database provides built-in security, continuous backups, and integrations with other AWS services. Neptune Global Database offers cross-region data replication for low-latency reads and writes, disaster recovery, and scalability for globally distributed applications.

Neptune Analytics is an analytics database engine for quickly analyzing large volumes of graph data to get insights and find trends from data stored in Amazon S3 buckets or a Neptune database. Neptune Analytics uses built-in algorithms, vector search, and in-memory computing to run queries on data with tens of billions of relationships in seconds. With Neptune Analytics, you can get insights and find trends by processing large amounts of graph data in seconds. To analyze graph data quickly and easily, Neptune Analytics stores large graph datasets in memory. It supports a library of optimized graph analytic algorithms, low-latency graph queries, and vector search capabilities within graph traversals.

GraphRAG-Indexing

GraphRAG-Retrieval

Natural Language Querying(NLQ)

Natural Language Querying (NLQ) is an emerging technology that allows users to interact with databases and knowledge graphs using plain language instead of rigid query languages like SQL. NLQ systems use advanced natural language processing and machine learning models to understand the intent behind a user's query and translate it into the appropriate database operations. This provides a more intuitive and user-friendly way for non-technical users to access and analyze data.

One prominent query language designed for knowledge graphs is openCypher. openCypher is a declarative graph query language that allows for powerful graph pattern matching and traversal operations. openCypher queries express what information should be retrieved from the graph, rather than how to retrieve it, making it more accessible than lower-level graph APIs. The combination of NLQ and openCypher is a compelling approach for enabling flexible, natural interactions with complex connected data.

However, it's important to recognize the inherent nature of large language models (LLMs) - they will not always generate the correct query, even if the original natural language question was clear. When the generated query does not produce the expected results, we'll need to employ below techniques such as query validation, prompt tuning, and query rewriting to troubleshoot and refine the process and to mitigate some of the challenges you are likely to run into when using LLMs to generate natural language queries.

• Prompt engineering - Allows us to control the instructions sent to the LLM to get around some of the differences between public examples and the syntax supported by Neptune.
• Schema validation - One common mistake that LLMs make when generating queries is that they can provide an incorrect edge direction, which leads to the query returning incorrect results. Schema validation compares the generated query against the graph schema and attempts to correct any issues with edge direction.
• Query error rewriting - One of the strengths of LLMs is their adaptability. When a query errors, due to syntax issues with the generated query, we pass the generated query and the error message back to the LLM and have it try to rewrite the query.

By combining these techniques together we can start to improve our query generation capabilities. However, we are still giving customers direct access to our graph so additional process safeguards, such as limiting users to read only access, need to be included to prevent misuse.

LLM Frameworks

• LangChain

LangChain is an open source framework for building Generative AI applications and provides key components to combine Large Language Models (LLMs) with external data.

Amazon Neptune with LangChain's NeptuneOpenCypherQAChain and Amazon Bedrock for the LLM.

LangChain's NeptuneOpenCypherQAChain provides a powerful way to enable natural language querying on top of a graph database like Amazon Neptune. The process works as follows:

• The user provides a natural language question.
• The system takes that question and combines it with an understanding of the schema (structure) of the underlying graph database.
• It then uses a large language model (LLM) to generate an appropriate openCypher query based on the user's question and the graph schema.
• The generated openCypher query is executed directly against the Neptune graph database.
• Finally, the results of the query are returned to the user, either directly or after being summarized by another language model.

This approach allows non-technical users to interact with complex graph data using plain language questions, without needing to learn the specifics of the openCypher query language. The LLM handles the translation from natural language to the appropriate graph traversal and pattern matching operations. This makes it easier for a wider range of users to derive insights from highly connected datasets stored in a graph database like Amazon Neptune.

• LlamaIndex

LlamaIndex is a data structure and tooling designed to create and interact with large language model indexes. It facilitates the storage, searching, and querying of textual data using advanced vector database techniques in conjunction with large language models like GPT. This enables efficient and effective retrieval of relevant information from extensive text corpora.

LlamaIndex is the Framework for Context-Augmented LLM Applications and LlamaIndex imposes no restriction on how you use LLMs. You can use LLMs as auto-complete, chatbots, agents, and more. It just makes using them easier.

Below are the Tools in Llama Index

• Data connectors ingest your existing data from their native source and format. These could be APIs, PDFs, SQL, and (much) more.
• Data indexes structure your data in intermediate representations that are easy and performant for LLMs to consume.
• Engines provide natural language access to your data. For example: Query engines are powerful interfaces for question-answering (e.g. a RAG flow). Chat engines are conversational interfaces for multi-message, "back and forth" interactions with your data.
• Agents are LLM-powered knowledge workers augmented by tools, from simple helper functions to API integrations and more.
• Observability/Evaluation integrations that enable you to rigorously experiment, evaluate, and monitor your app in a virtuous cycle.
• Workflows allow you to combine all of the above into an event-driven system far more flexible than other, graph-based approaches.

GraphRAG-Usage in Network and IT operations

Graph databases can be used to map complex network relationships, visualize network topologies, and perform root-cause analysis for outages

Graph databases can help telecommunications businesses become more agile by leveraging their connected data

Telecommunications is all about connections. Graph databases are, therefore, an excellent fit for modeling, storing and querying telecommunications data of all kinds. Whether managing increasingly complex network structures, ever-more-diverse product lines and bundles, or customer satisfaction and retention in today’s competitive environments, graph databases enable businesses to become more agile by leveraging their connected data.

From the network graph, to the social graph, to the call center graph, and the master data graph, telcos around the world have begun to use graph databases to achieve competitive advantage. Neo4j provides thousand-fold performance improvements and massive agility benefits over relational databases, enabling new levels of performance and insight.

Telecom inventory management tracks detailed information about physical and logical network components, including, for example, cell towers, cables, switches, routers, antennas and network devices.

References

Referred some existing Blogs and added few points based on my requirement .