Graph RAG: knowledge graph combined with LLM retrieval enhancement

12.23 Yuan Chuanghui·Shanghai Station, let’s talk about LLM infrastructure

This article reveals the Graph RAG method first proposed by NebulaGraph. This latest technology stack combines knowledge graphs and graph databases as large models with private knowledge systems. It is the third article in the LLM+ series, plus the previous graph context learning, Text2Cypher. Two articles, there are currently 3 articles related to NebulaGraph + LLM.

Graph RAG

In the first blog about contextual learning we introduced RAG (Retrieval Argumented Generation) based on specific tasks/problems In the document retrieval paradigm, we usually first collect the necessary context, and then use a machine learning model with cognitive capabilities to perform context learning (in-context learning) to synthesize the answer to the task.

With the help of LLM, a perception layer that can flexibly handle complex problems by just "speaking", you can build an intelligent application based on private knowledge in just two steps:

Retrieve relevant knowledge from a given document using various search methods such as Embedding and vector databases.
Understand and intelligently synthesize answers using LLM.

In this blog, we combine the latest exploration progress and thinking to try to compare Graph RAG with other methods to explain it more clearly. Additionally, we decided to start using the name Graph RAG to describe it.

Actually, Graph RAG was the first live seminar by Jerry Liu and I As mentioned in the Twitter Thread discussing and related discussions, similar content I < a i=7>NebulaGraph Community Live Broadcast was also introduced in Chinese.

The value of knowledge graphs in RAG

We explained this part in the first article. For example, a query: "Tell me everything about Apple and Steve Jobs" is a question and answer based on the autobiography of Steve Jobs. The context involved in this question is distributed in the autobiography. When the book has 30 pages (divided into blocks), the traditional "split data, Embedding and then vector search" method of using TOP-K to search in multiple document blocks is difficult to obtain such scattered, fine-grained and complete information. Furthermore, this approach can easily miss interrelated document chunks, resulting in incomplete information retrieval.

In addition, at a subsequent technical meeting, I had the honor to discuss with Xu Xu from leadscloud.com (they had The technical background of knowledge graph has also made similar explorations and attempts as ours!), which made me realize that knowledge graph can reduce the inaccuracy caused by embedding-based semantic search >. An interesting example given by Xu Xu is "insulation greenhouse" and "insulation cup". Although the two are semantically related, in most scenarios, the correlation under this general semantics (Embedding) Often it is something we don't want to create, and thus introduces "illusion" as the wrong context.

At this time, the knowledge graph that retains domain knowledge is a very direct means to alleviate and eliminate this illusion.

Implementing Graph RAG with NebulaGraph

A simple Graph RAG can be simply implemented as follows:

Use an LLM (or other) model to extract key entities from the problem
Retrieve subgraphs based on these entities, drilling down to a certain depth (e.g., 2)
Use the obtained context to generate answers using LLM.

The corresponding pseudocode might be like this:

# 伪代码

def _get_key_entities(query_str, llm=None ,with_llm=True):
    ...
    return _expand_synonyms(entities)

def _retrieve_subgraph_context(entities, depth=2, limit=30):
    ...
    return nebulagraph_store.get_relations(entities, depth, limit)

def _synthesize_answer(query_str, graph_rag_context, llm):
    return llm.predict(PROMPT_SYNTHESIZE_AND_REFINE, query_str, graph_rag_context)

def simple_graph_rag(query_str, nebulagraph_store, llm):
    entities = _get_key_entities(query_str, llm)
    graph_rag_context = _retrieve_subgraph_context(entities)
    return _synthesize_answer(
        query_str, graph_rag_context, llm)

However, with convenient LLM orchestration tools like LlamaIndex, developers can focus on the orchestration logic and pipeline design of LLM without having to deal with many detailed abstractions and implementations themselves.

So, using LlamaIndex, we can easily build Graph RAG and even integrate more complex RAG logic, such as Graph + Vector RAG.

In LlamaIndex, we have two ways to implement Graph RAG:

KnowledgeGraphIndexIt is only used to build a knowledge graph from scratch for any private data (based on LLM or other language models), and then perform Graph RAG with 4 lines of code:

graph_store = NebulaGraphStore(
    space_name=space_name,
    edge_types=edge_types,
    rel_prop_names=rel_prop_names,
    tags=tags,
)
storage_context = StorageContext.from_defaults(graph_store=graph_store)

# Build KG
kg_index = KnowledgeGraphIndex.from_documents(
    documents,
    storage_context=storage_context,
    max_triplets_per_chunk=10,
    space_name=space_name,
    edge_types=edge_types,
    rel_prop_names=rel_prop_names,
    tags=tags,
)

kg_query_engine = kg_index.as_query_engine()

KnowledgeGraphRAGQueryEngine Graph RAG can be performed on any existing knowledge graph. I haven't finished this PR yet, though.

graph_store = NebulaGraphStore(
    space_name=space_name,
    edge_types=edge_types,
    rel_prop_names=rel_prop_names,
    tags=tags,
)
storage_context = StorageContext.from_defaults(graph_store=graph_store)

graph_rag_query_engine = KnowledgeGraphRAGQueryEngine(
    storage_context=storage_context,
)

Finally, I made a Streamlit Demo to compare Graph RAG and Vector RAG. From this we can see that Graph RAG has not replaced Embedding and vector search methods, but enhance/supplement its shortcomings.

Text2Cypher

Another interesting approach to graph-based LLM is Text2Cypher. This method does not rely on subgraph retrieval of entities, but translates the task/question into an answer-oriented specific graph query, which is essentially the same as the Text2SQL method we often say.

Text2Cypher on NebulaGraph

We have already introduced in previous articles that thanks to LLM, implementing Text2Cypher is simpler and cheaper than traditional ML methods.

For example,LangChain: NebulaGraphQAChain andLlama Index: KnowledgeGraphQueryEngine Let's run Text2Cypher in 3 lines of code.

Compare Text2Cypher and (Sub)Graph RAG

The two methods differ mainly in their retrieval mechanisms. Text2Cypher generates graph schema queries based on the KG's Schema and a given task, while SubGraph RAG fetches relevant subgraphs to provide context.

Both have their advantages. In order for everyone to understand their characteristics more intuitively, I made this Demo video:

We can see that there are very clear differences in the visualization of the two graph query modes.

Graph RAG combined with Text2Cypher

However, there is no absolute good or bad between them. They each have their own advantages and disadvantages in different scenarios.

In the real world, we may not always know which method is more effective (to distinguish which one should be used), so I tend to consider utilizing both, so that the two retrieved results serve as context, one It is probably best to work together to generate the final answer.

The specific implementation method can already be achieved in this PR, you only need to set with_text2cypher=True, Graph The RAG will contain the Text2Cypher context, so stay tuned for its merging.

in conclusion

By integrating knowledge graphs and graph storage into the LLM technology stack, Graph RAG pushes RAG's context learning to a new level. It can extract fine-grained, fine-tuned, domain-specific and interconnected knowledge in LLM applications by leveraging existing (or newly created) knowledge graphs.

Stay tuned for deeper exploration and further developments in the world of graphs and LLM.