On Riemannian Optimization over Positive Definite Matrices with the Bures-Wasserstein Geometry

• URL: http://arxiv.org/abs/2106.00286v1
• Date: Tue, 1 Jun 2021 07:39:19 GMT
• Title: On Riemannian Optimization over Positive Definite Matrices with the Bures-Wasserstein Geometry
• Authors: Andi Han, Bamdev Mishra, Pratik Jawanpuria, Junbin Gao
• Abstract summary: We comparatively analyze the Bures-Wasserstein (BW) geometry with the popular Affine-Invariant (AI) geometry. We build on an observation that the BW metric has a linear dependence on SPD matrices in contrast to the quadratic dependence of the AI metric. We show that the BW geometry has a non-negative curvature, which further improves convergence rates of algorithms over the non-positively curved AI geometry.
• Score: 45.1944007785671
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we comparatively analyze the Bures-Wasserstein (BW) geometry with the popular Affine-Invariant (AI) geometry for Riemannian optimization on the symmetric positive definite (SPD) matrix manifold. Our study begins with an observation that the BW metric has a linear dependence on SPD matrices in contrast to the quadratic dependence of the AI metric. We build on this to show that the BW metric is a more suitable and robust choice for several Riemannian optimization problems over ill-conditioned SPD matrices. We show that the BW geometry has a non-negative curvature, which further improves convergence rates of algorithms over the non-positively curved AI geometry. Finally, we verify that several popular cost functions, which are known to be geodesic convex under the AI geometry, are also geodesic convex under the BW geometry. Extensive experiments on various applications support our findings.

Related papers

• RMLR: Extending Multinomial Logistic Regression into General Geometries [64.16104856124029]
  Our framework only requires minimal geometric properties, thus exhibiting broad applicability. We develop five families of SPD MLRs under five types of power-deformed metrics. On rotation matrices we propose Lie MLR based on the popular bi-invariant metric.
  arXiv  Detail & Related papers  (2024-09-28T18:38:21Z)
• Multiplicative Updates for Online Convex Optimization over Symmetric Cones [28.815822236291392]
  We introduce the Symmetric-Cone Multiplicative Weights Update (SCMWU), a projection-free algorithm for online optimization over the trace-one slice of an arbitrary symmetric cone. We show that SCMWU is equivalent to Follow-the-Regularized-Leader and Online Mirror Descent with symmetric-cone negative entropy as regularizer.
  arXiv  Detail & Related papers  (2023-07-06T17:06:43Z)
• Differential geometry with extreme eigenvalues in the positive semidefinite cone [1.9116784879310025]
  We present a route to a scalable geometric framework for the analysis and processing of SPD-valued data based on the efficient of extreme generalized eigenvalues. We define a novel iterative mean of SPD matrices based on this geometry and prove its existence and uniqueness for a given finite collection of points.
  arXiv  Detail & Related papers  (2023-04-14T18:37:49Z)
• Adaptive Log-Euclidean Metrics for SPD Matrix Learning [73.12655932115881]
  We propose Adaptive Log-Euclidean Metrics (ALEMs), which extend the widely used Log-Euclidean Metric (LEM) The experimental and theoretical results demonstrate the merit of the proposed metrics in improving the performance of SPD neural networks.
  arXiv  Detail & Related papers  (2023-03-26T18:31:52Z)
• Riemannian Optimization for Variance Estimation in Linear Mixed Models [0.0]
  We take a completely novel view on parameter estimation in linear mixed models by exploiting the intrinsic geometry of the parameter space. Our approach yields a higher quality of the variance parameter estimates compared to existing approaches.
  arXiv  Detail & Related papers  (2022-12-18T13:08:45Z)
• On a class of geodesically convex optimization problems solved via Euclidean MM methods [50.428784381385164]
  We show how a difference of Euclidean convexization functions can be written as a difference of different types of problems in statistics and machine learning. Ultimately, we helps the broader broader the broader the broader the broader the work.
  arXiv  Detail & Related papers  (2022-06-22T23:57:40Z)
• First-Order Algorithms for Min-Max Optimization in Geodesic Metric Spaces [93.35384756718868]
  min-max algorithms have been analyzed in the Euclidean setting. We prove that the extraiteient (RCEG) method corrected lastrate convergence at a linear rate.
  arXiv  Detail & Related papers  (2022-06-04T18:53:44Z)
• Geometry-aware Bayesian Optimization in Robotics using Riemannian Mat\'ern Kernels [64.62221198500467]
  We show how to implement geometry-aware kernels for Bayesian optimization. This technique can be used for control parameter tuning, parametric policy adaptation, and structure design in robotics.
  arXiv  Detail & Related papers  (2021-11-02T09:47:22Z)
• On Geometric Connections of Embedded and Quotient Geometries in Riemannian Fixed-rank Matrix Optimization [5.876141028192136]
  This paper proposes a general procedure for establishing the geometric landscape connections of a Riemannian optimization problem under the embedded and quotient geometries. We observe an algorithmic connection between two geometries with some specific Riemannian metrics in fixed-rank matrix optimization. Results provide a few new theoretical insights to unanswered questions in the literature.
  arXiv  Detail & Related papers  (2021-10-23T03:13:56Z)
• Learning with symmetric positive definite matrices via generalized Bures-Wasserstein geometry [40.23168342389821]
  We propose a novel generalization of the Bures-Wasserstein geometry, which we call the GBW geometry. We provide a rigorous treatment to study various differential geometric notions on the proposed novel generalized geometry. We also present experiments that illustrate the efficacy of the proposed GBW geometry over the BW geometry.
  arXiv  Detail & Related papers  (2021-10-20T10:03:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.