HypRAG: Hyperbolic Dense Retrieval for Retrieval Augmented Generation

• URL: http://arxiv.org/abs/2602.07739v1
• Date: Sun, 08 Feb 2026 00:18:05 GMT
• Title: HypRAG: Hyperbolic Dense Retrieval for Retrieval Augmented Generation
• Authors: Hiren Madhu, Ngoc Bui, Ali Maatouk, Leandros Tassiulas, Smita Krishnaswamy, Menglin Yang, Sukanta Ganguly, Kiran Srinivasan, Rex Ying,
• Abstract summary: HyTE-FH and HyTE-H are hyperbolic representations projecting pre-trained Euclidean embeddings into hyperbolic space.<n>The Outward Einstein Midpoint is a geometry-aware pooling operator that provably preserves hierarchical structure.<n>Our analysis also reveals that hyperbolic representations encode document specificity through norm-based separation.
• Score: 30.711145880382286
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Embedding geometry plays a fundamental role in retrieval quality, yet dense retrievers for retrieval-augmented generation (RAG) remain largely confined to Euclidean space. However, natural language exhibits hierarchical structure from broad topics to specific entities that Euclidean embeddings fail to preserve, causing semantically distant documents to appear spuriously similar and increasing hallucination risk. To address these limitations, we introduce hyperbolic dense retrieval, developing two model variants in the Lorentz model of hyperbolic space: HyTE-FH, a fully hyperbolic transformer, and HyTE-H, a hybrid architecture projecting pre-trained Euclidean embeddings into hyperbolic space. To prevent representational collapse during sequence aggregation, we introduce the Outward Einstein Midpoint, a geometry-aware pooling operator that provably preserves hierarchical structure. On MTEB, HyTE-FH outperforms equivalent Euclidean baselines, while on RAGBench, HyTE-H achieves up to 29% gains over Euclidean baselines in context relevance and answer relevance using substantially smaller models than current state-of-the-art retrievers. Our analysis also reveals that hyperbolic representations encode document specificity through norm-based separation, with over 20% radial increase from general to specific concepts, a property absent in Euclidean embeddings, underscoring the critical role of geometric inductive bias in faithful RAG systems.

Related papers

• HyperbolicRAG: Enhancing Retrieval-Augmented Generation with Hyperbolic Representations [11.678218711095269]
  Graph-based RAG enables large language models to access external knowledge.<n>We propose HyperbolicRAG, a retrieval framework that integrates hyperbolic geometry into graph-based RAG.
  arXiv  Detail & Related papers  (2025-11-24T06:27:58Z)
• Dense Semantic Matching with VGGT Prior [49.42199006453071]
  We propose an approach that retains VGGT's intrinsic strengths by reusing early feature stages, fine-tuning later ones, and adding a semantic head for bidirectional correspondences.<n>Our approach achieves superior geometry awareness, matching reliability, and manifold preservation, outperforming previous baselines.
  arXiv  Detail & Related papers  (2025-09-25T14:56:11Z)
• HLFormer: Enhancing Partially Relevant Video Retrieval with Hyperbolic Learning [77.434558721499]
  Partially Relevant Video Retrieval (PRVR) addresses the challenge of matching untrimmed videos with text queries describing only partial content.<n>We propose the first hyperbolic modeling framework for PRVR, namely HLFormer, which leverages hyperbolic space learning to compensate for the suboptimal hierarchical modeling capabilities of Euclidean space.
  arXiv  Detail & Related papers  (2025-07-23T10:59:46Z)
• Hyperbolic Category Discovery [12.654574453480635]
  Generalized Category Discovery (GCD) aims to categorize all images in the unlabelled subset, regardless of whether they belong to known or unknown classes.<n>Hyperbolic framework for learning hierarchy-aware representations and classifiers for GCD is presented.
  arXiv  Detail & Related papers  (2025-04-08T15:12:33Z)
• Achieving Hyperbolic-Like Expressiveness with Arbitrary Euclidean Regions: A New Approach to Hierarchical Embeddings [9.614222676567385]
  We present RegD, a flexible Euclidean framework that supports the use of arbitrary geometric regions as embedding representations.<n>RegD achieves hyperbolic-like expressiveness by incorporating a depth-based dissimilarity between regions, enabling it to emulate key properties of hyperbolic geometry.<n>Our empirical evaluation on diverse real-world datasets shows consistent performance gains over state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-01-29T09:44:03Z)
• From Semantics to Hierarchy: A Hybrid Euclidean-Tangent-Hyperbolic Space Model for Temporal Knowledge Graph Reasoning [1.1372536310854844]
  Temporal knowledge graph (TKG) reasoning predicts future events based on historical data. Existing Euclidean models excel at capturing semantics but struggle with hierarchy. We propose a novel hybrid geometric space approach that leverages the strengths of both Euclidean and hyperbolic models.
  arXiv  Detail & Related papers  (2024-08-30T10:33:08Z)
• HRCF: Enhancing Collaborative Filtering via Hyperbolic Geometric Regularization [52.369435664689995]
  We introduce a textitHyperbolic Regularization powered Collaborative Filtering (HRCF) and design a geometric-aware hyperbolic regularizer. Specifically, the proposal boosts optimization procedure via the root alignment and origin-aware penalty. Our proposal is able to tackle the over-smoothing problem caused by hyperbolic aggregation and also brings the models a better discriminative ability.
  arXiv  Detail & Related papers  (2022-04-18T06:11:44Z)
• Hyperbolic Neural Networks++ [66.16106727715061]
  We generalize the fundamental components of neural networks in a single hyperbolic geometry model, namely, the Poincar'e ball model. Experiments show the superior parameter efficiency of our methods compared to conventional hyperbolic components, and stability and outperformance over their Euclidean counterparts.
  arXiv  Detail & Related papers  (2020-06-15T08:23:20Z)
• Hyperbolic Manifold Regression [33.40757136529844]
  We consider the problem of performing manifold-valued regression onto an hyperbolic space as an intermediate component for a number of relevant machine learning applications. We propose a novel perspective on two challenging tasks: 1) hierarchical classification via label embeddings and 2) taxonomy extension of hyperbolic representations. Our experiments show that the strategy of leveraging the hyperbolic geometry is promising.
  arXiv  Detail & Related papers  (2020-05-28T10:16:30Z)
• APo-VAE: Text Generation in Hyperbolic Space [116.11974607497986]
  In this paper, we investigate text generation in a hyperbolic latent space to learn continuous hierarchical representations. An Adrial Poincare Variversaational Autoencoder (APo-VAE) is presented, where both the prior and variational posterior of latent variables are defined over a Poincare ball via wrapped normal distributions. Experiments in language modeling and dialog-response generation tasks demonstrate the winning effectiveness of the proposed APo-VAE model.
  arXiv  Detail & Related papers  (2020-04-30T19:05:41Z)
• Differentiating through the Fr\'echet Mean [51.32291896926807]
  Fr'echet mean is a generalization of the Euclidean mean. We show how to differentiate through the Fr'echet mean for arbitrary Riemannian manifold. This fully integrates the Fr'echet mean into the hyperbolic neural network pipeline.
  arXiv  Detail & Related papers  (2020-02-29T19:49:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.