Augmenting Textual Generation via Topology Aware Retrieval

• URL: http://arxiv.org/abs/2405.17602v1
• Date: Mon, 27 May 2024 19:02:18 GMT
• Title: Augmenting Textual Generation via Topology Aware Retrieval
• Authors: Yu Wang, Nedim Lipka, Ruiyi Zhang, Alexa Siu, Yuying Zhao, Bo Ni, Xin Wang, Ryan Rossi, Tyler Derr,
• Abstract summary: We develop a Topology-aware Retrieval-augmented Generation framework. This framework includes a retrieval module that selects texts based on their topological relationships. We have curated established text-attributed networks and conducted comprehensive experiments to validate the effectiveness of this framework.
• Score: 30.933176170660683
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite the impressive advancements of Large Language Models (LLMs) in generating text, they are often limited by the knowledge contained in the input and prone to producing inaccurate or hallucinated content. To tackle these issues, Retrieval-augmented Generation (RAG) is employed as an effective strategy to enhance the available knowledge base and anchor the responses in reality by pulling additional texts from external databases. In real-world applications, texts are often linked through entities within a graph, such as citations in academic papers or comments in social networks. This paper exploits these topological relationships to guide the retrieval process in RAG. Specifically, we explore two kinds of topological connections: proximity-based, focusing on closely connected nodes, and role-based, which looks at nodes sharing similar subgraph structures. Our empirical research confirms their relevance to text relationships, leading us to develop a Topology-aware Retrieval-augmented Generation framework. This framework includes a retrieval module that selects texts based on their topological relationships and an aggregation module that integrates these texts into prompts to stimulate LLMs for text generation. We have curated established text-attributed networks and conducted comprehensive experiments to validate the effectiveness of this framework, demonstrating its potential to enhance RAG with topological awareness.

Related papers

• Retrieval Augmented Generation of Literature-derived Polymer Knowledge: The Example of a Biodegradable Polymer Expert System [4.222675210976564]
  Polymer literature contains a large and growing body of experimental knowledge.<n>Much of it is buried in unstructured text and inconsistent terminology.<n>Existing tools typically extract narrow, study-specific facts in isolation.
  arXiv  Detail & Related papers  (2026-02-18T17:46:09Z)
• Disco-RAG: Discourse-Aware Retrieval-Augmented Generation [81.53888908988756]
  We propose Disco-RAG, a discourse-aware framework that injects discourse signals into the generation process.<n>Our method constructs intra-chunk discourse trees to capture local hierarchies and builds inter-chunk rhetorical graphs to model cross-passage coherence.<n>Experiments on question answering and long-document summarization benchmarks show the efficacy of our approach.
  arXiv  Detail & Related papers  (2026-01-07T20:32:50Z)
• Enhancing Retrieval-Augmented Generation with Topic-Enriched Embeddings: A Hybrid Approach Integrating Traditional NLP Techniques [0.0]
  This work proposes topic-enriched embeddings that integrate term-based signals and topic structure with contextual sentence embeddings.<n>By jointly capturing term-level and topic-level semantics, topic-enriched embeddings improve semantic clustering, increase retrieval precision, and reduce computational burden.
  arXiv  Detail & Related papers  (2025-12-31T13:43:57Z)
• A Survey on Retrieval And Structuring Augmented Generation with Large Language Models [29.707181003761004]
  Large Language Models (LLMs) have revolutionized natural language processing with their remarkable capabilities in text generation and reasoning.<n>However, these models face critical challenges when deployed in real-world applications, including outdated knowledge, and limited domain expertise.<n>Retrieval And Structuring (RAS) Augmented Generation addresses these limitations by integrating dynamic information retrieval with structured knowledge representations.
  arXiv  Detail & Related papers  (2025-09-12T21:25:25Z)
• Cross-Granularity Hypergraph Retrieval-Augmented Generation for Multi-hop Question Answering [49.43814054718318]
  Multi-hop question answering (MHQA) requires integrating knowledge scattered across multiple passages to derive the correct answer.<n>Traditional retrieval-augmented generation (RAG) methods primarily focus on coarse-grained textual semantic similarity.<n>We propose a novel RAG approach called HGRAG for MHQA that achieves cross-granularity integration of structural and semantic information via hypergraphs.
  arXiv  Detail & Related papers  (2025-08-15T06:36:13Z)
• A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Models [71.66119575697458]
  parallel text generation techniques aimed at breaking the token-by-token generation bottleneck and improving inference efficiency.<n>We categorize existing approaches into AR-based and Non-AR-based paradigms, and provide a detailed examination of the core techniques within each category.<n>We highlight recent advancements, identify open challenges, and outline promising directions for future research in parallel text generation.
  arXiv  Detail & Related papers  (2025-08-12T07:56:04Z)
• Enhancing Retrieval Augmented Generation with Hierarchical Text Segmentation Chunking [0.9968037829925942]
  This paper proposes a novel framework that enhances RAG by integrating hierarchical text segmentation and clustering.<n>During inference, the framework retrieves information by leveraging both segment-level and cluster-level vector representations.<n> Evaluations on the NarrativeQA, QuALITY, and QASPER datasets indicate that the proposed method achieved improved results compared to traditional chunking techniques.
  arXiv  Detail & Related papers  (2025-07-14T05:21:58Z)
• DISRetrieval: Harnessing Discourse Structure for Long Document Retrieval [51.89673002051528]
  DISRetrieval is a novel hierarchical retrieval framework that leverages linguistic discourse structure to enhance long document understanding.<n>Our studies confirm that discourse structure significantly enhances retrieval effectiveness across different document lengths and query types.
  arXiv  Detail & Related papers  (2025-05-26T14:45:12Z)
• Learning Interpretable Representations Leads to Semantically Faithful EEG-to-Text Generation [52.51005875755718]
  We focus on EEG-to-text decoding and address its hallucination issue through the lens of posterior collapse.<n>Acknowledging the underlying mismatch in information capacity between EEG and text, we reframe the decoding task as semantic summarization of core meanings.<n>Experiments on the public ZuCo dataset demonstrate that GLIM consistently generates fluent, EEG-grounded sentences.
  arXiv  Detail & Related papers  (2025-05-21T05:29:55Z)
• Pseudo-Knowledge Graph: Meta-Path Guided Retrieval and In-Graph Text for RAG-Equipped LLM [8.941718961724984]
  Pseudo-Knowledge Graph (PKG) framework integrates Meta-path Retrieval, In-graph Text and Vector Retrieval into Large Language Models. PKG offers a richer knowledge representation and improves accuracy in information retrieval.
  arXiv  Detail & Related papers  (2025-03-01T02:39:37Z)
• Mixture of Structural-and-Textual Retrieval over Text-rich Graph Knowledge Bases [78.62158923194153]
  Text-rich Graph Knowledge Bases (TG-KBs) have become increasingly crucial for answering queries by providing textual and structural knowledge. We propose a Mixture of Structural-and-Textual Retrieval (MoR) to retrieve these two types of knowledge via a Planning-Reasoning-Organizing framework.
  arXiv  Detail & Related papers  (2025-02-27T17:42:52Z)
• Think&Cite: Improving Attributed Text Generation with Self-Guided Tree Search and Progress Reward Modeling [63.98194996746229]
  Large language models (LLMs) are prone to hallucination and producing factually incorrect information. We propose a novel framework, called Think&Cite, and formulate attributed text generation as a multi-step reasoning problem integrated with search.
  arXiv  Detail & Related papers  (2024-12-19T13:55:48Z)
• Graph-Augmented Relation Extraction Model with LLMs-Generated Support Document [7.0421339410165045]
  This study introduces a novel approach to sentence-level relation extraction (RE) It integrates Graph Neural Networks (GNNs) with Large Language Models (LLMs) to generate contextually enriched support documents. Our experiments, conducted on the CrossRE dataset, demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-10-30T20:48:34Z)
• ConTReGen: Context-driven Tree-structured Retrieval for Open-domain Long-form Text Generation [26.4086456393314]
  Long-form text generation requires coherent, comprehensive responses that address complex queries with both breadth and depth. Existing iterative retrieval-augmented generation approaches often struggle to delve deeply into each facet of complex queries. This paper introduces ConTReGen, a novel framework that employs a context-driven, tree-structured retrieval approach.
  arXiv  Detail & Related papers  (2024-10-20T21:17:05Z)
• Knowledge-Aware Query Expansion with Large Language Models for Textual and Relational Retrieval [49.42043077545341]
  We propose a knowledge-aware query expansion framework, augmenting LLMs with structured document relations from knowledge graph (KG) We leverage document texts as rich KG node representations and use document-based relation filtering for our Knowledge-Aware Retrieval (KAR)
  arXiv  Detail & Related papers  (2024-10-17T17:03:23Z)
• Unleashing the Potential of Text-attributed Graphs: Automatic Relation Decomposition via Large Language Models [31.443478448031886]
  RoSE (Relation-oriented Semantic Edge-decomposition) is a novel framework that decomposes the graph structure by analyzing raw text attributes. Our framework significantly enhances node classification performance across various datasets, with improvements of up to 16% on the Wisconsin dataset.
  arXiv  Detail & Related papers  (2024-05-28T20:54:47Z)
• Loops On Retrieval Augmented Generation (LoRAG) [0.0]
  Loops On Retrieval Augmented Generation (LoRAG) is a new framework designed to enhance the quality of retrieval-augmented text generation. The architecture integrates a generative model, a retrieval mechanism, and a dynamic loop module. LoRAG surpasses existing state-of-the-art models in terms of BLEU score, ROUGE score, and perplexity.
  arXiv  Detail & Related papers  (2024-03-18T15:19:17Z)
• How Well Do Text Embedding Models Understand Syntax? [50.440590035493074]
  The ability of text embedding models to generalize across a wide range of syntactic contexts remains under-explored. Our findings reveal that existing text embedding models have not sufficiently addressed these syntactic understanding challenges. We propose strategies to augment the generalization ability of text embedding models in diverse syntactic scenarios.
  arXiv  Detail & Related papers  (2023-11-14T08:51:00Z)
• DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text [73.68051228972024]
  Large Language Models (LLMs) have exhibited impressive generation capabilities, but they suffer from hallucinations when relying on their internal knowledge. Retrieval-augmented LLMs have emerged as a potential solution to ground LLMs in external knowledge.
  arXiv  Detail & Related papers  (2023-10-31T04:37:57Z)
• Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learning [51.90524745663737]
  A key innovation is our use of explanations as features, which can be used to boost GNN performance on downstream tasks. Our method achieves state-of-the-art results on well-established TAG datasets. Our method significantly speeds up training, achieving a 2.88 times improvement over the closest baseline on ogbn-arxiv.
  arXiv  Detail & Related papers  (2023-05-31T03:18:03Z)
• Knowledge Graph-Augmented Language Models for Knowledge-Grounded Dialogue Generation [58.65698688443091]
  We propose SUbgraph Retrieval-augmented GEneration (SURGE), a framework for generating context-relevant and knowledge-grounded dialogues with Knowledge Graphs (KGs) Our framework first retrieves the relevant subgraph from the KG, and then enforces consistency across facts by perturbing their word embeddings conditioned by the retrieved subgraph. We validate our SURGE framework on OpendialKG and KOMODIS datasets, showing that it generates high-quality dialogues that faithfully reflect the knowledge from KG.
  arXiv  Detail & Related papers  (2023-05-30T08:36:45Z)
• Knowledge-based Review Generation by Coherence Enhanced Text Planning [45.473253542837995]
  We propose a novel Coherence Enhanced Text Planning model (CETP) based on knowledge graphs (KGs) to improve both global and local coherence for review generation. For global coherence, we design a hierarchical self-attentive architecture with both subgraph- and node-level attention to enhance the correlations between subgraphs. Experiments on three datasets confirm the effectiveness of our model on improving the content coherence of generated texts.
  arXiv  Detail & Related papers  (2021-05-09T02:12:05Z)
• Positioning yourself in the maze of Neural Text Generation: A Task-Agnostic Survey [54.34370423151014]
  This paper surveys the components of modeling approaches relaying task impacts across various generation tasks such as storytelling, summarization, translation etc. We present an abstraction of the imperative techniques with respect to learning paradigms, pretraining, modeling approaches, decoding and the key challenges outstanding in the field in each of them.
  arXiv  Detail & Related papers  (2020-10-14T17:54:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.