Hypergraph-based Multimodal Retrieval-Augmented Generation with Incremental Refinement

• URL: http://arxiv.org/abs/2608.16628v1
• Date: Mon, 17 Aug 2026 14:30:09 GMT
• Title: Hypergraph-based Multimodal Retrieval-Augmented Generation with Incremental Refinement
• Abstract summary: Hyper-M2RAG is a novel framework that redefines multimodal document retrieval through High-order Hyper Representation Learning.<n>To mitigate semantic fragmentation caused by physical redundancy, we introduce an Anchor-driven Incremental Refinement mechanism.<n>This mechanism identifies boundary-crossing anchor nodes and reconstructs their local hyper-topology using one-hop neighborhood contexts.
• Score: 19.099989510112604
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Modern Multimodal Retrieval-Augmented Generation (M-RAG) systems are fundamentally limited by the binary connectivity paradigm of traditional simple graphs, which fails to capture the intricate, high-order correlations among heterogeneous entities, such as the N-ary relationships between a visual chart, its scattered textual descriptions, and underlying numerical data. Furthermore, existing refinement strategies often rely on exhaustive, full-page reconstruction to align cross-modal information, leading to prohibitive computational redundancy and the introduction of contextual noise in long-form document processing. In this paper, we propose Hyper-M2RAG, a novel framework that redefines multimodal document retrieval through High-order Hypergraph Representation Learning. We first formalize the document structure as a Multimodal Hypergraph, utilizing hyperedges as unified semantic containers to encapsulate multi-way associations across text, images, and tables, thereby transcending point-to-point modeling. To mitigate semantic fragmentation caused by physical pagination, we introduce an Anchor-driven Incremental Refinement mechanism. Rather than performing a global sweep, our approach identifies boundary-crossing anchor nodes and reconstructs their local hyper-topology using one-hop neighborhood contexts. This targeted refinement effectively bridges cross-page knowledge gaps with minimal computational footprints. Extensive evaluations on multimodal benchmarking datasets demonstrate that Hyper-M2RAG significantly outperforms state-of-the-art methods in both retrieval precision and generation coherence. Our code is available at https://github.com/ShenAoChen2001/MMHRAG.

Related papers

• DualG-MRAG: Decoupling Macro-Reasoning and Micro-Matching for Multimodal Retrieval-Augmented Generation [23.80343378702536]
  Multimodal Retrieval-Augmented Generation (MM-RAG) has shown promising results, but it still struggles with complex multi-hop reasoning tasks.<n>We propose DualG-MRAG, a Dual-tier framework that introduces a decoupled architecture comprising Macro-reasoning and Micro-matching Graphs for Multimodal RAG.
  arXiv  Detail & Related papers  (2026-07-30T17:40:05Z)
• Empowering Cross-Domain Sequential Recommendation with Hybrid Tokenization and Serial-Parallel Decoding [63.74998371740272]
  Cross-domain sequential recommendation (CDSR) aims to model users' dynamic interest transitions and sequential patterns across multiple domains.<n>We propose GenCDSR, an effective and efficient generative framework for CDSR.<n>We show that GenCDSR achieves an average accuracy improvement of 1.5 percent and an average inference latency reduction of 85.1 percent compared with state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-07-21T08:01:15Z)
• MM-BizRAG: Rethinking Multimodal Retrieval-Augmented Generation for General Purpose Enterprise Q&A [13.040489131301081]
  MM-BizRAG proactively extracts and represents document structure via a document structure-aware split.<n> MM-BizRAG consistently outperforms state-of-the-art vision-centric baselines by up to 32% points.
  arXiv  Detail & Related papers  (2026-06-02T21:31:47Z)
• Hypergraph as Language [49.35033483821496]
  We propose a hypergraph-native alignment framework for large language models.<n>For hypergraph structures, existing methods often fail to preserve the native semantics that multiple objects are jointly connected by the same high-order relation.<n>We show that Hyper-Align significantly outperforms existing methods across in-domain and zero-shot evaluations.
  arXiv  Detail & Related papers  (2026-05-21T01:09:48Z)
• Robust Multimodal Recommendation via Graph Retrieval-Enhanced Modality Completion [71.20734649881258]
  Real-world multimodal datasets often suffer from modality incompleteness due to sensor failures, annotation scarcity, or privacy constraints.<n>One effective solution is modality completion, which reconstructs missing features to provide modality-complete graphs for downstream tasks.<n>We propose GRE-MC, a Graph Retrieval-Enhanced Modality Completion framework, to overcome these limitations.
  arXiv  Detail & Related papers  (2026-05-01T13:50:52Z)
• MetaHGNIE: Meta-Path Induced Hypergraph Contrastive Learning in Heterogeneous Knowledge Graphs [16.36978652807043]
  We propose MetaHGNIE, a meta-path induced hypergraph contrastive learning framework for disentangling and aligning structural and semantic information.<n>Our results highlight the effectiveness of explicitly modeling higher-order interactions and cross-modal alignment in heterogeneous knowledge graphs.
  arXiv  Detail & Related papers  (2025-12-13T22:21:33Z)
• Cog-RAG: Cognitive-Inspired Dual-Hypergraph with Theme Alignment Retrieval-Augmented Generation [43.737415090081555]
  We propose a theme-aligned dual-hypergraph RAG framework (Cog-RAG) that uses a theme hypergraph to capture inter-chunk thematic structure and an entity hypergraph to model high-order semantic relations.<n>Our experiments demonstrate that Cog-RAG significantly outperforms existing state-of-the-art baseline approaches.
  arXiv  Detail & Related papers  (2025-11-17T10:10:33Z)
• Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understanding [61.36285696607487]
  Document understanding is critical for applications from financial analysis to scientific discovery.<n>Current approaches, whether OCR-based pipelines feeding Large Language Models (LLMs) or native Multimodal LLMs (MLLMs) face key limitations.<n>Retrieval-Augmented Generation (RAG) helps ground models in external data, but documents' multimodal nature, combining text, tables, charts, and layout, demands a more advanced paradigm: Multimodal RAG.
  arXiv  Detail & Related papers  (2025-10-17T02:33:16Z)
• Cross-Granularity Hypergraph Retrieval-Augmented Generation for Multi-hop Question Answering [49.43814054718318]
  Multi-hop question answering (MHQA) requires integrating knowledge scattered across multiple passages to derive the correct answer.<n>Traditional retrieval-augmented generation (RAG) methods primarily focus on coarse-grained textual semantic similarity.<n>We propose a novel RAG approach called HGRAG for MHQA that achieves cross-granularity integration of structural and semantic information via hypergraphs.
  arXiv  Detail & Related papers  (2025-08-15T06:36:13Z)
• Hypergraph Transformer for Semi-Supervised Classification [50.92027313775934]
  We propose a novel hypergraph learning framework, HyperGraph Transformer (HyperGT) HyperGT uses a Transformer-based neural network architecture to effectively consider global correlations among all nodes and hyperedges. It achieves comprehensive hypergraph representation learning by effectively incorporating global interactions while preserving local connectivity patterns.
  arXiv  Detail & Related papers  (2023-12-18T17:50:52Z)
• Hypergraph Neural Networks through the Lens of Message Passing: A Common Perspective to Homophily and Architecture Design [6.993718963663415]
  We introduce a novel conceptualization of homophily in higher-order networks based on a Message Passing scheme.<n>We investigate some natural, yet mostly unexplored, strategies for processing higher-order structures within HNNs.
  arXiv  Detail & Related papers  (2023-10-11T17:35:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.