Intrinsic Muon: Spectral Optimization on Riemannian Matrix Manifolds

• URL: http://arxiv.org/abs/2605.09238v1
• Date: Sun, 10 May 2026 00:39:13 GMT
• Title: Intrinsic Muon: Spectral Optimization on Riemannian Matrix Manifolds
• Abstract summary: Muon and related norm-constrained matrix oracles have become central to large-scale learning problems.<n>We introduce intrinsic Muon (iMuon), a unified framework that yields closed-form updates on the fixed-rank, SPD, Stiefel, and Grassmann matrices.<n> Experiments on LoRA finetuning of LLMs, image classification, and subspace learning illustrate the proposed approach.
• Score: 14.996920125049002
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Muon and related norm-constrained matrix optimizers have become central to large-scale learning problems. They are formulated as a linear maximization oracle (LMO) over an ambient matrix-norm ball in unconstrained Euclidean space. However, these do not generalize cleanly to manifold-valued parameters such as low-rank factorizations, orthogonality constraints, or symmetric positive definite (SPD) matrices. Naively restricting the Muon LMO to the tangent space (i) breaks quotient symmetries and (ii) couples the tangent-space constraint with an ambient norm bound, thereby obstructing closed-form solutions on various manifolds of interest. We resolve both issues with a single observation: every Riemannian metric canonically lifts a unitarily invariant Euclidean norm to an intrinsic norm on each tangent space, and the resulting intrinsic norm constrained LMO is symmetry preserving. Building on this, we introduce intrinsic Muon (iMuon), a unified framework that yields closed-form updates on the fixed-rank, SPD, Stiefel, and Grassmann manifolds for any unitarily invariant norm, including the spectral, Frobenius, and nuclear norms. We establish convergence guarantees for both deterministic and stochastic iMuon with rate constants that depend only on the manifold dimension. Notably, on the fixed-rank manifold this constant depends only on the rank, making the rate independent of factor conditioning and removing the runtime factor-rescaling required by prior work. Experiments on LoRA finetuning of LLMs, image classification, and subspace learning illustrate the efficacy of the proposed approach.

Related papers

• Generalized nonparametric regression in reproducing kernel Hilbert spaces: Consistency and rates of convergence [0.0]
  We develop a comprehensive theory for M-lev in reproducing spaces.<n>For Sobo spaces we obtain new rates that connect to spaces of smoothness with mixed smoothness functions.
  arXiv  Detail & Related papers  (2026-06-22T08:12:31Z)
• Operational Tube-Sector Theory of Quantum State Distinguishability Under Generalized Symmetries [2.2003859300452198]
  A variational principle for quantum-state distinguishability is established in many-body systems with generalized symmetries.<n>When symmetry actions terminate at entanglement cuts, distinguishability is governed by boundary tube algebras.<n>The resulting optimal measurement structure is uniquely fixed by the center of the boundary tube algebra.
  arXiv  Detail & Related papers  (2026-06-18T01:02:07Z)
• LieBN: Batch Normalization over Lie Groups [75.83671298040186]
  We propose LieBN, a framework for Batch Normalization (RBN) over Lie groups.<n>We instantiate LieBN across nine distinct geometries.<n>We introduce a novel right-invariant metric and extend three existing Lie group structures.
  arXiv  Detail & Related papers  (2026-06-13T07:06:42Z)
• Muon Dynamics as a Spectral Wasserstein Flow [21.689846521201588]
  We introduce a family of Wasserstein norms on positive semidefinite matrices.<n>We prove the equivalence between the static and dynamic Benamou-Brenier formulations.<n>We identify exact gradientity associated with flow models.
  arXiv  Detail & Related papers  (2026-04-06T17:41:12Z)
• Rethinking Diffusion Models with Symmetries through Canonicalization with Applications to Molecular Graph Generation [56.361076943802594]
  CanonFlow achieves state-of-the-art performance on the challenging GEOM-DRUG dataset, and the advantage remains large in few-step generation.
  arXiv  Detail & Related papers  (2026-02-16T18:58:55Z)
• Manifold constrained steepest descent [0.0]
  We propose emphManifold Constrained Steepest Descent (MCSD), a single-loop framework for optimization over manifold.<n>We also introduce emphSPEL, the spectral specialization of MCSD on the Stiefel manifold.
  arXiv  Detail & Related papers  (2026-01-29T10:08:37Z)
• Graph-based Clustering Revisited: A Relaxation of Kernel $k$-Means Perspective [73.18641268511318]
  We propose a graph-based clustering algorithm that only relaxes the orthonormal constraint to derive clustering results.<n>To ensure a doubly constraint into a gradient, we transform the non-negative constraint into a class probability parameter.
  arXiv  Detail & Related papers  (2025-09-23T09:14:39Z)
• Absorb and Converge: Provable Convergence Guarantee for Absorbing Discrete Diffusion Models [66.0716790920952]
  We provide the first finite-time error bounds and convergence rate analysis for discrete diffusion models using absorbing rate matrices.<n>We establish the first convergence guarantees for both the $tau$-leaping and uniformization samplers under absorbing rate matrices.<n>Under suitable assumptions, we provide convergence guarantees without early stopping.
  arXiv  Detail & Related papers  (2025-06-02T23:14:35Z)
• Intrinsic Regularization via Curved Momentum Space: A Geometric Solution to Divergences in Quantum Field Theory [0.0]
  UV divergences in quantum field theory (QFT) have long been a fundamental challenge.<n>We propose a novel and self-consistent approach in which UV regularization emerges naturally from the curved geometry of momentum space.<n>We show seamless extension to Minkowski space, maintainining regularization properties in relativistic QFT.
  arXiv  Detail & Related papers  (2025-02-20T10:49:26Z)
• Mirror Descent Under Generalized Smoothness [23.5387392871236]
  We introduce a new $ell*$-smoothness concept that measures the norm of Hessians in terms of a general norm and its dual.<n>We establish convergence for mirror-descent-type algorithms, matching the rates under the classic smoothness.
  arXiv  Detail & Related papers  (2025-02-02T11:23:10Z)
• Intrinsic Bayesian Cramér-Rao Bound with an Application to Covariance Matrix Estimation [49.67011673289242]
  This paper presents a new performance bound for estimation problems where the parameter to estimate lies in a smooth manifold. It induces a geometry for the parameter manifold, as well as an intrinsic notion of the estimation error measure.
  arXiv  Detail & Related papers  (2023-11-08T15:17:13Z)
• Optimal policy evaluation using kernel-based temporal difference methods [78.83926562536791]
  We use kernel Hilbert spaces for estimating the value function of an infinite-horizon discounted Markov reward process. We derive a non-asymptotic upper bound on the error with explicit dependence on the eigenvalues of the associated kernel operator. We prove minimax lower bounds over sub-classes of MRPs.
  arXiv  Detail & Related papers  (2021-09-24T14:48:20Z)
• Benign Overfitting of Constant-Stepsize SGD for Linear Regression [122.70478935214128]
  inductive biases are central in preventing overfitting empirically. This work considers this issue in arguably the most basic setting: constant-stepsize SGD for linear regression. We reflect on a number of notable differences between the algorithmic regularization afforded by (unregularized) SGD in comparison to ordinary least squares.
  arXiv  Detail & Related papers  (2021-03-23T17:15:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.