Symplectic Stiefel manifold: tractable metrics, second-order geometry and Newton's methods

• URL: http://arxiv.org/abs/2406.14299v1
• Date: Thu, 20 Jun 2024 13:26:06 GMT
• Title: Symplectic Stiefel manifold: tractable metrics, second-order geometry and Newton's methods
• Authors: Bin Gao, Nguyen Thanh Son, Tatjana Stykel,
• Abstract summary: We develop explicit second-order geometry and Newton's methods on the symplectic Stiefel manifold. We then solve the resulting Newton equation, as the central step of Newton's methods. Various numerical experiments are presented to validate the proposed methods.
• Score: 1.190653833745802
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Optimization under the symplecticity constraint is an approach for solving various problems in quantum physics and scientific computing. Building on the results that this optimization problem can be transformed into an unconstrained problem on the symplectic Stiefel manifold, we construct geometric ingredients for Riemannian optimization with a new family of Riemannian metrics called tractable metrics and develop Riemannian Newton schemes. The newly obtained ingredients do not only generalize several existing results but also provide us with freedom to choose a suitable metric for each problem. To the best of our knowledge, this is the first try to develop the explicit second-order geometry and Newton's methods on the symplectic Stiefel manifold. For the Riemannian Newton method, we first consider novel operator-valued formulas for computing the Riemannian Hessian of a~cost function, which further allows the manifold to be endowed with a weighted Euclidean metric that can provide a preconditioning effect. We then solve the resulting Newton equation, as the central step of Newton's methods, directly via transforming it into a~saddle point problem followed by vectorization, or iteratively via applying any matrix-free iterative method either to the operator Newton equation or its saddle point formulation. Finally, we propose a hybrid Riemannian Newton optimization algorithm that enjoys both global convergence and quadratic/superlinear local convergence at the final stage. Various numerical experiments are presented to validate the proposed methods.

Related papers

• Riemannian stochastic optimization methods avoid strict saddle points [68.80251170757647]
  We show that policies under study avoid strict saddle points / submanifolds with probability 1. This result provides an important sanity check as it shows that, almost always, the limit state of an algorithm can only be a local minimizer.
  arXiv  Detail & Related papers  (2023-11-04T11:12:24Z)
• Linearization Algorithms for Fully Composite Optimization [61.20539085730636]
  This paper studies first-order algorithms for solving fully composite optimization problems convex compact sets. We leverage the structure of the objective by handling differentiable and non-differentiable separately, linearizing only the smooth parts.
  arXiv  Detail & Related papers  (2023-02-24T18:41:48Z)
• Unified Convergence Theory of Stochastic and Variance-Reduced Cubic Newton Methods [37.1630298053787]
  We propose a new framework, which we call the helper framework. It provides a unified view of the variance and second-order algorithms equipped with global complexity guarantees.
  arXiv  Detail & Related papers  (2023-02-23T12:18:28Z)
• Faster Riemannian Newton-type Optimization by Subsampling and Cubic Regularization [3.867143522757309]
  This work is on constrained large-scale non-constrained optimization where the constraint set implies a manifold structure. We propose a new second-order saddleian optimization algorithm, aiming at improving convergence and reducing computational cost.
  arXiv  Detail & Related papers  (2023-02-22T00:37:44Z)
• Robust empirical risk minimization via Newton's method [9.797319790710711]
  A new variant of Newton's method for empirical risk minimization is studied. The gradient and Hessian of the objective function are replaced by robust estimators. An algorithm for obtaining robust Newton directions based on the conjugate gradient method is also proposed.
  arXiv  Detail & Related papers  (2023-01-30T18:54:54Z)
• The Dynamics of Riemannian Robbins-Monro Algorithms [101.29301565229265]
  We propose a family of Riemannian algorithms generalizing and extending the seminal approximation framework of Robbins and Monro. Compared to their Euclidean counterparts, Riemannian algorithms are much less understood due to lack of a global linear structure on the manifold. We provide a general template of almost sure convergence results that mirrors and extends the existing theory for Euclidean Robbins-Monro schemes.
  arXiv  Detail & Related papers  (2022-06-14T12:30:11Z)
• Augmented Newton Method for Optimization: Global Linear Rate and Momentum Interpretation [0.0]
  We propose two variants of Newton method for solving unconstrained problem. We generate novel variants of the Newton method namely the Penalty Newton method and the Augmented Newton method.
  arXiv  Detail & Related papers  (2022-05-23T04:34:10Z)
• On Riemannian Approach for Constrained Optimization Model in Extreme Classification Problems [2.7436792484073638]
  A constrained optimization problem is formulated as an optimization problem on matrix manifold. The proposed approach is tested on several real world large scale multi-label datasets.
  arXiv  Detail & Related papers  (2021-09-30T11:28:35Z)
• Newton-LESS: Sparsification without Trade-offs for the Sketched Newton Update [88.73437209862891]
  In second-order optimization, a potential bottleneck can be computing the Hessian matrix of the optimized function at every iteration. We show that the Gaussian sketching matrix can be drastically sparsified, significantly reducing the computational cost of sketching. We prove that Newton-LESS enjoys nearly the same problem-independent local convergence rate as Gaussian embeddings.
  arXiv  Detail & Related papers  (2021-07-15T17:33:05Z)
• Automatic differentiation for Riemannian optimization on low-rank matrix and tensor-train manifolds [71.94111815357064]
  In scientific computing and machine learning applications, matrices and more general multidimensional arrays (tensors) can often be approximated with the help of low-rank decompositions. One of the popular tools for finding the low-rank approximations is to use the Riemannian optimization.
  arXiv  Detail & Related papers  (2021-03-27T19:56:00Z)
• Bayesian Quadrature on Riemannian Data Manifolds [79.71142807798284]
  A principled way to model nonlinear geometric structure inherent in data is provided. However, these operations are typically computationally demanding. In particular, we focus on Bayesian quadrature (BQ) to numerically compute integrals over normal laws. We show that by leveraging both prior knowledge and an active exploration scheme, BQ significantly reduces the number of required evaluations.
  arXiv  Detail & Related papers  (2021-02-12T17:38:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.