HyperCore: The Core Framework for Building Hyperbolic Foundation Models with Comprehensive Modules

• URL: http://arxiv.org/abs/2504.08912v1
• Date: Fri, 11 Apr 2025 18:35:46 GMT
• Title: HyperCore: The Core Framework for Building Hyperbolic Foundation Models with Comprehensive Modules
• Authors: Neil He, Menglin Yang, Rex Ying,
• Abstract summary: We introduce HyperCore, a comprehensive open-source framework that provides core modules for constructing hyperbolic foundation models.<n>To demonstrate its versatility, we build and test the first fully hyperbolic vision transformers (LViT) with a fine-tuning pipeline.<n>Our experiments demonstrate that LViT outperforms its Euclidean counterpart.
• Score: 15.257990326035694
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Hyperbolic neural networks have emerged as a powerful tool for modeling hierarchical data across diverse modalities. Recent studies show that token distributions in foundation models exhibit scale-free properties, suggesting that hyperbolic space is a more suitable ambient space than Euclidean space for many pre-training and downstream tasks. However, existing tools lack essential components for building hyperbolic foundation models, making it difficult to leverage recent advancements. We introduce HyperCore, a comprehensive open-source framework that provides core modules for constructing hyperbolic foundation models across multiple modalities. HyperCore's modules can be effortlessly combined to develop novel hyperbolic foundation models, eliminating the need to extensively modify Euclidean modules from scratch and possible redundant research efforts. To demonstrate its versatility, we build and test the first fully hyperbolic vision transformers (LViT) with a fine-tuning pipeline, the first fully hyperbolic multimodal CLIP model (L-CLIP), and a hybrid Graph RAG with a hyperbolic graph encoder. Our experiments demonstrate that LViT outperforms its Euclidean counterpart. Additionally, we benchmark and reproduce experiments across hyperbolic GNNs, CNNs, Transformers, and vision Transformers to highlight HyperCore's advantages.

Related papers

• Machine Unlearning in Hyperbolic vs. Euclidean Multimodal Contrastive Learning: Adapting Alignment Calibration to MERU [50.9588132578029]
  This paper investigates machine unlearning in hyperbolic contrastive learning.<n>We adapt Alignment to MERU, a model that embeds images and text in hyperbolic space to better capture semantic hierarchies.<n>Our approach introduces hyperbolic-specific components including entailment calibration and norm regularization that leverage the unique properties of hyperbolic space.
  arXiv  Detail & Related papers  (2025-03-19T12:47:37Z)
• Hypformer: Exploring Efficient Hyperbolic Transformer Fully in Hyperbolic Space [47.4014545166959]
  We introduce Hypformer, a novel hyperbolic Transformer based on the Lorentz model of hyperbolic geometry. We develop a linear self-attention mechanism in hyperbolic space, enabling hyperbolic Transformer to process billion-scale graph data and long-sequence inputs for the first time.
  arXiv  Detail & Related papers  (2024-07-01T13:44:38Z)
• Hyperbolic Delaunay Geometric Alignment [52.835250875177756]
  We propose a similarity score for comparing datasets in a hyperbolic space. The core idea is counting the edges of the hyperbolic Delaunay graph connecting datapoints across the given sets. We provide an empirical investigation on synthetic and real-life biological data and demonstrate that HyperDGA outperforms the hyperbolic version of classical distances between sets.
  arXiv  Detail & Related papers  (2024-04-12T17:14:58Z)
• Dynamic Hyperbolic Attention Network for Fine Hand-object Reconstruction [76.5549647815413]
  We propose the first precise hand-object reconstruction method in hyperbolic space, namely Dynamic Hyperbolic Attention Network (DHANet) Our method learns mesh features with rich geometry-image multi-modal information and models better hand-object interaction.
  arXiv  Detail & Related papers  (2023-09-06T13:00:10Z)
• Fully Hyperbolic Convolutional Neural Networks for Computer Vision [3.3964154468907486]
  We present HCNN, a fully hyperbolic convolutional neural network (CNN) designed for computer vision tasks. Based on the Lorentz model, we propose novel formulations of the convolutional layer, batch normalization, and multinomial logistic regression. Experiments on standard vision tasks demonstrate the promising performance of our HCNN framework in both hybrid and fully hyperbolic settings.
  arXiv  Detail & Related papers  (2023-03-28T12:20:52Z)
• A Unification Framework for Euclidean and Hyperbolic Graph Neural Networks [8.080621697426997]
  Hyperbolic neural networks can effectively capture the inherent hierarchy of graph datasets. They entangle multiple incongruent (gyro-)vector spaces within a layer, which makes them limited in terms of generalization and scalability. We propose the Poincare disk model as our search space, and apply all approximations on the disk. We demonstrate that our model not only leverages the power of Euclidean networks such as interpretability and efficient execution of various model components, but also outperforms both Euclidean and hyperbolic counterparts on various benchmarks.
  arXiv  Detail & Related papers  (2022-06-09T05:33:02Z)
• Enhancing Hyperbolic Graph Embeddings via Contrastive Learning [7.901082408569372]
  We propose a novel Hyperbolic Graph Contrastive Learning (HGCL) framework which learns node representations through multiple hyperbolic spaces. Experimental results on multiple real-world datasets demonstrate the superiority of the proposed HGCL.
  arXiv  Detail & Related papers  (2022-01-21T06:10:05Z)
• Fully Hyperbolic Neural Networks [63.22521652077353]
  We propose a fully hyperbolic framework to build hyperbolic networks based on the Lorentz model. We show that our method has better performance for building both shallow and deep networks.
  arXiv  Detail & Related papers  (2021-05-31T03:36:49Z)
• Hyperbolic Variational Graph Neural Network for Modeling Dynamic Graphs [77.33781731432163]
  We learn dynamic graph representation in hyperbolic space, for the first time, which aims to infer node representations. We present a novel Hyperbolic Variational Graph Network, referred to as HVGNN. In particular, to model the dynamics, we introduce a Temporal GNN (TGNN) based on a theoretically grounded time encoding approach.
  arXiv  Detail & Related papers  (2021-04-06T01:44:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.