GRAG: Graph Retrieval-Augmented Generation

• URL: http://arxiv.org/abs/2405.16506v2
• Date: Mon, 21 Oct 2024 00:55:13 GMT
• Title: GRAG: Graph Retrieval-Augmented Generation
• Authors: Yuntong Hu, Zhihan Lei, Zheng Zhang, Bo Pan, Chen Ling, Liang Zhao,
• Abstract summary: Graph Retrieval-Augmented Generation (GRAG) tackles the fundamental challenges in retrieving textual subgraphs. We propose a novel divide-and-conquer strategy that retrieves the optimal subgraph structure in linear time. Our approach significantly outperforms current state-of-the-art RAG methods.
• Score: 14.98084919101233
• License:
• Abstract: Naive Retrieval-Augmented Generation (RAG) focuses on individual documents during retrieval and, as a result, falls short in handling networked documents which are very popular in many applications such as citation graphs, social media, and knowledge graphs. To overcome this limitation, we introduce Graph Retrieval-Augmented Generation (GRAG), which tackles the fundamental challenges in retrieving textual subgraphs and integrating the joint textual and topological information into Large Language Models (LLMs) to enhance its generation. To enable efficient textual subgraph retrieval, we propose a novel divide-and-conquer strategy that retrieves the optimal subgraph structure in linear time. To achieve graph context-aware generation, incorporate textual graphs into LLMs through two complementary views-the text view and the graph view-enabling LLMs to more effectively comprehend and utilize the graph context. Extensive experiments on graph reasoning benchmarks demonstrate that in scenarios requiring multi-hop reasoning on textual graphs, our GRAG approach significantly outperforms current state-of-the-art RAG methods.

Related papers

• Exploring Graph Structure Comprehension Ability of Multimodal Large Language Models: Case Studies [7.067145619709089]
  This study investigates the impact of graph visualisations on Large Language Models (LLMs) performance. Our experiments compare the effectiveness of multimodal approaches against purely textual graph representations.
  arXiv  Detail & Related papers  (2024-09-13T14:26:58Z)
• Hierarchical Compression of Text-Rich Graphs via Large Language Models [63.75293588479027]
  Text-rich graphs are prevalent in data mining contexts like e-commerce and academic graphs. This paper introduces Hierarchical Compression'' (HiCom), a novel method to align the capabilities of LLMs with the structure of text-rich graphs. HiCom can outperform both GNNs and LLM backbones for node classification on e-commerce and citation graphs.
  arXiv  Detail & Related papers  (2024-06-13T07:24:46Z)
• Parameter-Efficient Tuning Large Language Models for Graph Representation Learning [62.26278815157628]
  We introduce Graph-aware. Efficient Fine-Tuning - GPEFT, a novel approach for efficient graph representation learning. We use a graph neural network (GNN) to encode structural information from neighboring nodes into a graph prompt. We validate our approach through comprehensive experiments conducted on 8 different text-rich graphs, observing an average improvement of 2% in hit@1 and Mean Reciprocal Rank (MRR) in link prediction evaluations.
  arXiv  Detail & Related papers  (2024-04-28T18:36:59Z)
• G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answering [61.93058781222079]
  We develop a flexible question-answering framework targeting real-world textual graphs. We introduce the first retrieval-augmented generation (RAG) approach for general textual graphs. G-Retriever performs RAG over a graph by formulating this task as a Prize-Collecting Steiner Tree optimization problem.
  arXiv  Detail & Related papers  (2024-02-12T13:13:04Z)
• When Graph Data Meets Multimodal: A New Paradigm for Graph Understanding and Reasoning [54.84870836443311]
  The paper presents a new paradigm for understanding and reasoning about graph data by integrating image encoding and multimodal technologies. This approach enables the comprehension of graph data through an instruction-response format, utilizing GPT-4V's advanced capabilities. The study evaluates this paradigm on various graph types, highlighting the model's strengths and weaknesses, particularly in Chinese OCR performance and complex reasoning tasks.
  arXiv  Detail & Related papers  (2023-12-16T08:14:11Z)
• Which Modality should I use -- Text, Motif, or Image? : Understanding Graphs with Large Language Models [14.251972223585765]
  This paper introduces a new approach to encoding a graph with diverse modalities, such as text, image, and motif, and prompts to approximate a graph's global connectivity. The study also presents GraphTMI, a novel benchmark for evaluating Large Language Models (LLMs) in graph structure analysis.
  arXiv  Detail & Related papers  (2023-11-16T12:45:41Z)
• GRENADE: Graph-Centric Language Model for Self-Supervised Representation Learning on Text-Attributed Graphs [22.282756544376493]
  We develop a novel Graph-Centric Language model, GRENADE, to solve the problem of self-supervised representation learning on text-attributed graphs. GRENADE exploits the synergistic effect of both pre-trained language model and graph neural network by optimizing with two specialized self-supervised learning algorithms. The proposed graph-centric self-supervised learning algorithms effectively help GRENADE to capture informative textual semantics as well as structural context information on text-attributed graphs.
  arXiv  Detail & Related papers  (2023-10-23T17:18:35Z)
• Improving Graph-Based Text Representations with Character and Word Level N-grams [30.699644290131044]
  We propose a new word-character text graph that combines word and character n-gram nodes together with document nodes. We also propose two new graph-based neural models, WCTextGCN and WCTextGAT, for modeling our proposed text graph.
  arXiv  Detail & Related papers  (2022-10-12T08:07:54Z)
• Graph Pooling for Graph Neural Networks: Progress, Challenges, and Opportunities [128.55790219377315]
  Graph neural networks have emerged as a leading architecture for many graph-level tasks. graph pooling is indispensable for obtaining a holistic graph-level representation of the whole graph.
  arXiv  Detail & Related papers  (2022-04-15T04:02:06Z)
• Promoting Graph Awareness in Linearized Graph-to-Text Generation [72.83863719868364]
  We study the ability of linearized models to encode local graph structures. Our findings motivate solutions to enrich the quality of models' implicit graph encodings. We find that these denoising scaffolds lead to substantial improvements in downstream generation in low-resource settings.
  arXiv  Detail & Related papers  (2020-12-31T18:17:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.