Alignment and Outer Shell Isotropy for Hyperbolic Graph Contrastive Learning

• URL: http://arxiv.org/abs/2310.18209v1
• Date: Fri, 27 Oct 2023 15:31:42 GMT
• Title: Alignment and Outer Shell Isotropy for Hyperbolic Graph Contrastive Learning
• Authors: Yifei Zhang, Hao Zhu, Jiahong Liu, Piotr Koniusz, Irwin King
• Abstract summary: We propose a novel contrastive learning framework to learn high-quality graph embedding. Specifically, we design the alignment metric that effectively captures the hierarchical data-invariant information. We show that in the hyperbolic space one has to address the leaf- and height-level uniformity which are related to properties of trees.
• Score: 69.6810940330906
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning good self-supervised graph representations that are beneficial to downstream tasks is challenging. Among a variety of methods, contrastive learning enjoys competitive performance. The embeddings of contrastive learning are arranged on a hypersphere that enables the Cosine distance measurement in the Euclidean space. However, the underlying structure of many domains such as graphs exhibits highly non-Euclidean latent geometry. To this end, we propose a novel contrastive learning framework to learn high-quality graph embedding. Specifically, we design the alignment metric that effectively captures the hierarchical data-invariant information, as well as we propose a substitute of uniformity metric to prevent the so-called dimensional collapse. We show that in the hyperbolic space one has to address the leaf- and height-level uniformity which are related to properties of trees, whereas in the ambient space of the hyperbolic manifold, these notions translate into imposing an isotropic ring density towards boundaries of Poincar\'e ball. This ring density can be easily imposed by promoting the isotropic feature distribution on the tangent space of manifold. In the experiments, we demonstrate the efficacy of our proposed method across different hyperbolic graph embedding techniques in both supervised and self-supervised learning settings.

Related papers

• Improving embedding of graphs with missing data by soft manifolds [51.425411400683565]
  The reliability of graph embeddings depends on how much the geometry of the continuous space matches the graph structure. We introduce a new class of manifold, named soft manifold, that can solve this situation. Using soft manifold for graph embedding, we can provide continuous spaces to pursue any task in data analysis over complex datasets.
  arXiv  Detail & Related papers  (2023-11-29T12:48:33Z)
• Hyperbolic vs Euclidean Embeddings in Few-Shot Learning: Two Sides of the Same Coin [49.12496652756007]
  We show that the best few-shot results are attained for hyperbolic embeddings at a common hyperbolic radius. In contrast to prior benchmark results, we demonstrate that better performance can be achieved by a fixed-radius encoder equipped with the Euclidean metric.
  arXiv  Detail & Related papers  (2023-09-18T14:51:46Z)
• Hyperbolic Contrastive Learning [12.170564544949308]
  We propose a novel contrastive learning framework to learn semantic relationships in the hyperbolic space. We show that our proposed method achieves better results on self-supervised pretraining, supervised classification, and higher robust accuracy than baseline methods.
  arXiv  Detail & Related papers  (2023-02-02T20:47:45Z)
• Geometry Contrastive Learning on Heterogeneous Graphs [50.58523799455101]
  This paper proposes a novel self-supervised learning method, termed as Geometry Contrastive Learning (GCL) GCL views a heterogeneous graph from Euclidean and hyperbolic perspective simultaneously, aiming to make a strong merger of the ability of modeling rich semantics and complex structures. Extensive experiments on four benchmarks data sets show that the proposed approach outperforms the strong baselines.
  arXiv  Detail & Related papers  (2022-06-25T03:54:53Z)
• HRCF: Enhancing Collaborative Filtering via Hyperbolic Geometric Regularization [52.369435664689995]
  We introduce a textitHyperbolic Regularization powered Collaborative Filtering (HRCF) and design a geometric-aware hyperbolic regularizer. Specifically, the proposal boosts optimization procedure via the root alignment and origin-aware penalty. Our proposal is able to tackle the over-smoothing problem caused by hyperbolic aggregation and also brings the models a better discriminative ability.
  arXiv  Detail & Related papers  (2022-04-18T06:11:44Z)
• Enhancing Hyperbolic Graph Embeddings via Contrastive Learning [7.901082408569372]
  We propose a novel Hyperbolic Graph Contrastive Learning (HGCL) framework which learns node representations through multiple hyperbolic spaces. Experimental results on multiple real-world datasets demonstrate the superiority of the proposed HGCL.
  arXiv  Detail & Related papers  (2022-01-21T06:10:05Z)
• Hyperbolic Graph Embedding with Enhanced Semi-Implicit Variational Inference [48.63194907060615]
  We build off of semi-implicit graph variational auto-encoders to capture higher-order statistics in a low-dimensional graph latent representation. We incorporate hyperbolic geometry in the latent space through a Poincare embedding to efficiently represent graphs exhibiting hierarchical structure.
  arXiv  Detail & Related papers  (2020-10-31T05:48:34Z)
• Graph Geometry Interaction Learning [41.10468385822182]
  We develop a novel Geometry Interaction Learning (GIL) method for graphs, a well-suited and efficient alternative for learning abundant geometric properties in graph. Our method endows each node the freedom to determine the importance of each geometry space via a flexible dual feature interaction learning and probability assembling mechanism. Promising experimental results are presented for five benchmark datasets on node classification and link prediction tasks.
  arXiv  Detail & Related papers  (2020-10-23T02:40:28Z)
• Ultrahyperbolic Representation Learning [13.828165530602224]
  In machine learning, data is usually represented in a (flat) Euclidean space where distances between points are along straight lines. We propose a representation living on a pseudo-Riemannian manifold of constant nonzero curvature. We provide the necessary learning tools in this geometry and extend gradient-based optimization techniques.
  arXiv  Detail & Related papers  (2020-07-01T03:49:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.