Learning to Normalize on the SPD Manifold under Bures-Wasserstein Geometry

• URL: http://arxiv.org/abs/2504.00660v1
• Date: Tue, 01 Apr 2025 11:12:58 GMT
• Title: Learning to Normalize on the SPD Manifold under Bures-Wasserstein Geometry
• Authors: Rui Wang, Shaocheng Jin, Ziheng Chen, Xiaoqing Luo, Xiao-Jun Wu,
• Abstract summary: Covariance matrices have proven highly effective across many scientific fields.<n>The primary challenge in representation learning is to respect this underlying geometric structure.<n>We propose a novel RBN algorithm based on the Bures-Wasserstein metric, incorporating a learnable metric parameter.
• Score: 11.846361701184254
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Covariance matrices have proven highly effective across many scientific fields. Since these matrices lie within the Symmetric Positive Definite (SPD) manifold - a Riemannian space with intrinsic non-Euclidean geometry, the primary challenge in representation learning is to respect this underlying geometric structure. Drawing inspiration from the success of Euclidean deep learning, researchers have developed neural networks on the SPD manifolds for more faithful covariance embedding learning. A notable advancement in this area is the implementation of Riemannian batch normalization (RBN), which has been shown to improve the performance of SPD network models. Nonetheless, the Riemannian metric beneath the existing RBN might fail to effectively deal with the ill-conditioned SPD matrices (ICSM), undermining the effectiveness of RBN. In contrast, the Bures-Wasserstein metric (BWM) demonstrates superior performance for ill-conditioning. In addition, the recently introduced Generalized BWM (GBWM) parameterizes the vanilla BWM via an SPD matrix, allowing for a more nuanced representation of vibrant geometries of the SPD manifold. Therefore, we propose a novel RBN algorithm based on the GBW geometry, incorporating a learnable metric parameter. Moreover, the deformation of GBWM by matrix power is also introduced to further enhance the representational capacity of GBWM-based RBN. Experimental results on different datasets validate the effectiveness of our proposed method.

Related papers

• RMLR: Extending Multinomial Logistic Regression into General Geometries [64.16104856124029]
  Our framework only requires minimal geometric properties, thus exhibiting broad applicability. We develop five families of SPD MLRs under five types of power-deformed metrics. On rotation matrices we propose Lie MLR based on the popular bi-invariant metric.
  arXiv  Detail & Related papers  (2024-09-28T18:38:21Z)
• Riemannian Self-Attention Mechanism for SPD Networks [34.794770395408335]
  An SPD manifold self-attention mechanism (SMSA) is proposed in this paper. An SMSA-based geometric learning module (SMSA-GL) is designed for the sake of improving the discrimination of structured representations.
  arXiv  Detail & Related papers  (2023-11-28T12:34:46Z)
• Riemannian Multinomial Logistics Regression for SPD Neural Networks [60.11063972538648]
  We propose a new type of deep neural network for Symmetric Positive Definite (SPD) matrices. Our framework offers a novel intrinsic explanation for the most popular LogEig classifier in existing SPD networks. The effectiveness of our method is demonstrated in three applications: radar recognition, human action recognition, and electroencephalography (EEG) classification.
  arXiv  Detail & Related papers  (2023-05-18T20:12:22Z)
• Adaptive Log-Euclidean Metrics for SPD Matrix Learning [73.12655932115881]
  We propose Adaptive Log-Euclidean Metrics (ALEMs), which extend the widely used Log-Euclidean Metric (LEM) The experimental and theoretical results demonstrate the merit of the proposed metrics in improving the performance of SPD neural networks.
  arXiv  Detail & Related papers  (2023-03-26T18:31:52Z)
• Manifold Gaussian Variational Bayes on the Precision Matrix [70.44024861252554]
  We propose an optimization algorithm for Variational Inference (VI) in complex models. We develop an efficient algorithm for Gaussian Variational Inference whose updates satisfy the positive definite constraint on the variational covariance matrix. Due to its black-box nature, MGVBP stands as a ready-to-use solution for VI in complex models.
  arXiv  Detail & Related papers  (2022-10-26T10:12:31Z)
• Deep Optimal Transport for Domain Adaptation on SPD Manifolds [9.552869120136005]
  We introduce a new geometric deep learning framework that combines optimal transport theory with the geometry of SPD manifold. Our approach aligns data distributions while respecting the manifold structure, effectively reducing both marginal and conditional discrepancies. We validate our method on three cross-session brain computer interface datasets, KU, BNCI2014001, and BNCI2015001.
  arXiv  Detail & Related papers  (2022-01-15T03:13:02Z)
• On Riemannian Optimization over Positive Definite Matrices with the Bures-Wasserstein Geometry [45.1944007785671]
  We comparatively analyze the Bures-Wasserstein (BW) geometry with the popular Affine-Invariant (AI) geometry. We build on an observation that the BW metric has a linear dependence on SPD matrices in contrast to the quadratic dependence of the AI metric. We show that the BW geometry has a non-negative curvature, which further improves convergence rates of algorithms over the non-positively curved AI geometry.
  arXiv  Detail & Related papers  (2021-06-01T07:39:19Z)
• Learning Log-Determinant Divergences for Positive Definite Matrices [47.61701711840848]
  In this paper, we propose to learn similarity measures in a data-driven manner. We capitalize on the alphabeta-log-det divergence, which is a meta-divergence parametrized by scalars alpha and beta. Our key idea is to cast these parameters in a continuum and learn them from data.
  arXiv  Detail & Related papers  (2021-04-13T19:09:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.