Hyperbolic Busemann Learning with Ideal Prototypes

• URL: http://arxiv.org/abs/2106.14472v1
• Date: Mon, 28 Jun 2021 08:36:59 GMT
• Title: Hyperbolic Busemann Learning with Ideal Prototypes
• Authors: Mina Ghadimi Atigh, Martin Keller-Ressel, Pascal Mettes
• Abstract summary: In this work, we propose Hyperbolic Busemann Learning for representation learning of arbitrary data. To be able to compute proximities to ideal prototypes, we introduce the penalised Busemann loss. Empirically, we show that our approach provides a natural interpretation of classification confidence, while outperforming recent hyperspherical and hyperbolic prototype approaches.
• Score: 14.525985704735055
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Hyperbolic space has become a popular choice of manifold for representation learning of arbitrary data, from tree-like structures and text to graphs. Building on the success of deep learning with prototypes in Euclidean and hyperspherical spaces, a few recent works have proposed hyperbolic prototypes for classification. Such approaches enable effective learning in low-dimensional output spaces and can exploit hierarchical relations amongst classes, but require privileged information about class labels to position the hyperbolic prototypes. In this work, we propose Hyperbolic Busemann Learning. The main idea behind our approach is to position prototypes on the ideal boundary of the Poincare ball, which does not require prior label knowledge. To be able to compute proximities to ideal prototypes, we introduce the penalised Busemann loss. We provide theory supporting the use of ideal prototypes and the proposed loss by proving its equivalence to logistic regression in the one-dimensional case. Empirically, we show that our approach provides a natural interpretation of classification confidence, while outperforming recent hyperspherical and hyperbolic prototype approaches.

Related papers

• HyperPg -- Prototypical Gaussians on the Hypersphere for Interpretable Deep Learning [2.0599237172837523]
  ProtoPNet learn, which part of a test image "look like" known prototypical parts from training images, combines predictive power with the inherent interpretability of case-based reasoning. This work introduces HyperPg, a new prototype representation leveraging Gaussian distributions on a hypersphere in latent space. experiments on CUB-200-2011 and Stanford Cars datasets demonstrate that HyperPgNet outperforms other prototype learning architectures.
  arXiv  Detail & Related papers  (2024-10-11T15:50:31Z)
• A Coding-Theoretic Analysis of Hyperspherical Prototypical Learning Geometry [25.514947992281378]
  Hyperspherical Prototypical Learning (HPL) is a supervised approach to representation learning that designs class prototypes on the unit hypersphere. Previous approaches to HPL have either of the following shortcomings: (i) they follow an unprincipled optimisation procedure; or (ii) they are theoretically sound, but are constrained to only one possible latent dimension.
  arXiv  Detail & Related papers  (2024-07-10T13:44:19Z)
• HMSN: Hyperbolic Self-Supervised Learning by Clustering with Ideal Prototypes [7.665392786787577]
  We use hyperbolic representation space for self-supervised representation learning for prototype-based clustering approaches. We extend the Masked Siamese Networks to operate on the Poincar'e ball model of hyperbolic space. Unlike previous methods we project to the hyperbolic space at the output of the encoder network and utilise a hyperbolic projection head to ensure that the representations used for downstream tasks remain hyperbolic.
  arXiv  Detail & Related papers  (2023-05-18T12:38:40Z)
• Automatically Discovering Novel Visual Categories with Self-supervised Prototype Learning [68.63910949916209]
  This paper tackles the problem of novel category discovery (NCD), which aims to discriminate unknown categories in large-scale image collections. We propose a novel adaptive prototype learning method consisting of two main stages: prototypical representation learning and prototypical self-training. We conduct extensive experiments on four benchmark datasets and demonstrate the effectiveness and robustness of the proposed method with state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-08-01T16:34:33Z)
• HRCF: Enhancing Collaborative Filtering via Hyperbolic Geometric Regularization [52.369435664689995]
  We introduce a textitHyperbolic Regularization powered Collaborative Filtering (HRCF) and design a geometric-aware hyperbolic regularizer. Specifically, the proposal boosts optimization procedure via the root alignment and origin-aware penalty. Our proposal is able to tackle the over-smoothing problem caused by hyperbolic aggregation and also brings the models a better discriminative ability.
  arXiv  Detail & Related papers  (2022-04-18T06:11:44Z)
• BMD: A General Class-balanced Multicentric Dynamic Prototype Strategy for Source-free Domain Adaptation [74.93176783541332]
  Source-free Domain Adaptation (SFDA) aims to adapt a pre-trained source model to the unlabeled target domain without accessing the well-labeled source data. To make up for the absence of source data, most existing methods introduced feature prototype based pseudo-labeling strategies. We propose a general class-Balanced Multicentric Dynamic prototype strategy for the SFDA task.
  arXiv  Detail & Related papers  (2022-04-06T13:23:02Z)
• Rethinking Semantic Segmentation: A Prototype View [126.59244185849838]
  We present a nonparametric semantic segmentation model based on non-learnable prototypes. Our framework yields compelling results over several datasets. We expect this work will provoke a rethink of the current de facto semantic segmentation model design.
  arXiv  Detail & Related papers  (2022-03-28T21:15:32Z)
• Dual Prototypical Contrastive Learning for Few-shot Semantic Segmentation [55.339405417090084]
  We propose a dual prototypical contrastive learning approach tailored to the few-shot semantic segmentation (FSS) task. The main idea is to encourage the prototypes more discriminative by increasing inter-class distance while reducing intra-class distance in prototype feature space. We demonstrate that the proposed dual contrastive learning approach outperforms state-of-the-art FSS methods on PASCAL-5i and COCO-20i datasets.
  arXiv  Detail & Related papers  (2021-11-09T08:14:50Z)
• A Theory of Hyperbolic Prototype Learning [0.0]
  We introduce Hyperbolic Prototype Learning, where class labels are represented by ideal points (points at infinity) in hyperbolic space. Learning is achieved by minimizing the 'penalized Busemann loss', a new loss function based on the Busemann function of hyperbolic geometry.
  arXiv  Detail & Related papers  (2020-10-15T13:45:02Z)
• A Fully Hyperbolic Neural Model for Hierarchical Multi-Class Classification [7.8176853587105075]
  Hyperbolic spaces offer a mathematically appealing approach for learning hierarchical representations of symbolic data. This work proposes a fully hyperbolic model for multi-class multi-label classification, which performs all operations in hyperbolic space. A thorough analysis sheds light on the impact of each component in the final prediction and showcases its ease of integration with Euclidean layers.
  arXiv  Detail & Related papers  (2020-10-05T14:42:56Z)
• Robust Large-Margin Learning in Hyperbolic Space [64.42251583239347]
  We present the first theoretical guarantees for learning a classifier in hyperbolic rather than Euclidean space. We provide an algorithm to efficiently learn a large-margin hyperplane, relying on the careful injection of adversarial examples. We prove that for hierarchical data that embeds well into hyperbolic space, the low embedding dimension ensures superior guarantees.
  arXiv  Detail & Related papers  (2020-04-11T19:11:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.