Move on Muon : A Hamiltonian probability gradient flow perspective of Muon optimizer

• URL: http://arxiv.org/abs/2605.23871v1
• Date: Fri, 22 May 2026 17:28:53 GMT
• Title: Move on Muon : A Hamiltonian probability gradient flow perspective of Muon optimizer
• Abstract summary: We develop a gradient flow on the space of probability measures defined on matrix-valued parameters induced by regularized Muon.<n>We prove an exact Hamiltonian dissipation identity, showing that the Hamiltonian energy decreases monotonically.<n>We extend the formulation to Hilbert-valued feature maps on product matrix spaces, yielding a blockwise Muon probability flow.
• Score: 6.003944904428636
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We develop a gradient flow on the space of probability measures defined on matrix-valued parameters induced by regularized Muon, an analytically smoothed version of the idealized Muon optimizer. The key observation is that the regularized orthogonalization map is the gradient of a smooth Fenchel-dual smoothing of the nuclear norm. This identifies the (regularized) Muon update as a mirror/prox step in the update variable, with momentum acting as the dual coordinate. We use this structure to lift Muon from a single matrix parameter to finite-particle probability objectives of the form $J(ρ)=R\left(\int F d ρ\right)$, a setting motivated by mean-field descriptions of neural-network training, and derive the inertial continuous-time limit. Using this structure, we derive the finite-particle continuous-time limit under the inertial scaling of step size and momentum, and then pass to a phase-space mean-field equation over probability laws on parameter-momentum pairs. The resulting flow can be shown to be a damped Hamiltonian probability dynamics whose kinetic energy is induced by the regularized Muon mirror potential. We prove an exact Hamiltonian dissipation identity, showing that the Hamiltonian energy decreases monotonically. While the target objective itself need not be monotone along the inertial Muon dynamics, under additional gradient-dominance, bounded-momentum, and curvature/alignment assumptions, we obtain continuous and discrete-time exponential convergence rates for the objective gap. We also study the well-posedness of the mean-field limit equation and establish propagation of chaos guarantees for the interacting particle system. Finally, we extend the formulation to Hilbert-valued feature maps on product matrix spaces, yielding a blockwise Muon probability flow applicable to smooth transformer mixture-of-experts models.

Related papers

• Can Quantum Field Theory be Recovered from Time-Symmetric Stochastic Mechanics? Part I: Generalizing the Liouville Equation [0.0]
  We explore whether quantum field theory can be understood as the statistical mechanics of a time-reversal-invariant generalization of Hamiltonian dynamics.<n>The question to what extent this equation can be interpreted in terms of objective field theories is discussed in a companion paper.
  arXiv  Detail & Related papers  (2026-03-20T18:19:21Z)
• Mousse: Rectifying the Geometry of Muon with Curvature-Aware Preconditioning [72.62839712454196]
  Recent advances in spectral optimization, notably Muon, have demonstrated that constraining update steps to the Stiefel manifold can significantly accelerate training and improveimation.<n>We argue that this "egalitarian" constraint is suboptimal for Deep Neural Networks, where the curvature spectrum is known to be highly heavy-tailed and ill-conditioned.<n>We propose textbfMousse (textbfMuon textbfOptimization textbfUtilizing textbfShampoo's textbfStruct
  arXiv  Detail & Related papers  (2026-03-10T14:03:49Z)
• Symmetry-protected topology and deconfined solitons in a multi-link $\mathbb{Z}_2$ gauge theory [45.88028371034407]
  We study a $mathbbZ$ lattice gauge theory defined on a multi-graph with links that can be visualized as great circles of a spherical shell.<n>We show that this leads to state-dependent tunneling amplitudes underlying a phenomenon analogous to the Peierls instability.<n>By performining a detailed analysis based on matrix product states, we prove that charge deconfinement emerges as a consequence of charge-fractionalization.
  arXiv  Detail & Related papers  (2026-03-02T22:59:25Z)
• Deep Sequence Modeling with Quantum Dynamics: Language as a Wave Function [0.0]
  We introduce a sequence modeling framework in which the latent state is a complex-valued wave function evolving on a finite-dimensional Hilbert space under a learned, time-dependent Hamiltonian.<n> Token probabilities are extracted using the Born rule, a quadratic measurement operator that couples magnitudes and relative phases.<n>We derive a continuity equation for the latent probability mass, yielding conserved pairwise currents that serve as a built-in diagnostic.
  arXiv  Detail & Related papers  (2026-02-24T23:42:18Z)
• Preconditioning Benefits of Spectral Orthogonalization in Muon [50.62925024212989]
  We study the effectiveness of a simplified variant of Muon in two case studies: matrix factorization and in-context learning of linear transformers.<n>Our analysis reveals that the Muon dynamics decouple into a collection of independent scalar sequences in the spectral domain, each exhibiting similar convergence behavior.
  arXiv  Detail & Related papers  (2026-01-20T00:08:31Z)
• Measurement-induced transitions for interacting fermions [43.04146484262759]
  We develop a field-theoretical framework that provides a unified approach to observables characterizing entanglement and charge fluctuations.<n>Within this framework, we derive a replicated Keldysh non-linear sigma model (NLSM)<n>By using the renormalization-group approach for the NLSM, we determine the phase diagram and the scaling of physical observables.
  arXiv  Detail & Related papers  (2024-10-09T18:00:08Z)
• Theory of free fermions dynamics under partial post-selected monitoring [49.1574468325115]
  We derive a partial post-selected Schrdinger"o equation based on a microscopic description of continuous weak measurement. We show that the passage to the monitored universality occurs abruptly at finite partial post-selection. Our approach establishes a way to study MiPTs for arbitrary subsets of quantum trajectories.
  arXiv  Detail & Related papers  (2023-12-21T16:53:42Z)
• Third quantization of open quantum systems: new dissipative symmetries and connections to phase-space and Keldysh field theory formulations [77.34726150561087]
  We reformulate the technique of third quantization in a way that explicitly connects all three methods. We first show that our formulation reveals a fundamental dissipative symmetry present in all quadratic bosonic or fermionic Lindbladians. For bosons, we then show that the Wigner function and the characteristic function can be thought of as ''wavefunctions'' of the density matrix.
  arXiv  Detail & Related papers  (2023-02-27T18:56:40Z)
• Birth-death dynamics for sampling: Global convergence, approximations and their asymptotics [9.011881058913184]
  We build a practical numerical system based on the pure-death dynamics. We show that the kernelized dynamics converge on finite time intervals, to pure gradient-death dynamics shrinks to zero. Finally we prove the long-time results on the convergence of the states of the kernelized dynamics towards the Gibbs measure.
  arXiv  Detail & Related papers  (2022-11-01T13:30:26Z)
• Lindblad master equations for quantum systems coupled to dissipative bosonic modes [0.0]
  We derive Lindblad master equations for a subsystem whose dynamics is coupled to bosonic modes. We apply this formalism to the dissipative Dicke model and derive a Lindblad master equation for the atomic spins. This master equation accurately predicts the Dicke phase transition and gives the correct steady state.
  arXiv  Detail & Related papers  (2022-03-07T11:21:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.