Momentum Stability and Adaptive Control in Stochastic Reconfiguration

• URL: http://arxiv.org/abs/2604.18357v1
• Date: Mon, 20 Apr 2026 14:49:30 GMT
• Title: Momentum Stability and Adaptive Control in Stochastic Reconfiguration
• Abstract summary: Variational Monte Carlo (VMC) combined with expressive neural network wavefunctions has become a powerful route to ground-state calculations.<n>We study efficient and stable wavefunction optimization via high-accuracy divergence via adaptive reconfiguration (SR)<n>We propose PRIME-SR, a tuning-free momentum-adaptive SR method based on effective spectral dimension and subspace overlap.
• Score: 6.526210062544706
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Variational Monte Carlo (VMC) combined with expressive neural network wavefunctions has become a powerful route to high-accuracy ground-state calculations, yet its practical success hinges on efficient and stable wavefunction optimization. While stochastic reconfiguration (SR) provides a geometry-aware preconditioner motivated by imaginary-time evolution, its Kaczmarz-inspired variant, subsampled projected-increment natural gradient descent (SPRING), achieves state-of-the-art empirical performance. However, the effectiveness of SPRING is highly sensitive to the choice of a momentum-like parameter $μ$. The original sensitivity of $μ$ and the instability observed at $μ=1$, have remained unclear. In this work, we clarify the distinct mechanisms governing the regimes $μ<1$ and $μ=1$. We establish convergence guarantees for $0\leμ<1$ under mild assumptions, and construct counterexamples showing that $μ=1$ can induce divergence via uncontrolled growth along kernel-related directions when the step-size is not summable. Motivated by these theoretical insights and numerical observations, we further propose \textit{Principal Range Informed MomEntum SR} (PRIME-SR), a tuning-free momentum-adaptive SR method based on effective spectral dimension and subspace overlap. PRIME-SR achieves performance comparable to optimally tuned SPRING while significantly improving robustness in VMC optimization.

Related papers

• When Does Scale-Invariant Optimization Become Unstable? An Exact Schedule Law with Weight Decay [6.625270757109195]
  We show that a single quantity captures all schedule and decay forcing, while norm growth induces an opposing geometric self-quenching effect.<n>This yields a sharp boundary that cleanly separates contraction- and expansion-dominated effective learning rate regimes.
  arXiv  Detail & Related papers  (2026-09-08T17:46:17Z)
• Distribution-Aware Robust Bilevel Optimization: Quantile-Guided Huber Updates in Two-Timescale Stochastic Approximation [16.106678611834102]
  Bilevel optimization (BLO) is fundamental to hierarchical decision-making but suffers from critical instability under heavy-tailed noise.<n>Existing techniques typically rely on magnitude checks, which fail to distinguish informative geometric signals from outliers.
  arXiv  Detail & Related papers  (2026-06-21T10:57:37Z)
• Universally Empowering Zeroth-Order Optimization via Adaptive Layer-wise Sampling [43.822941944402544]
  Zeroth-Order optimization presents a promising memory-efficient paradigm for fine-tuning Large Language Models.<n>However, its practical adoption is severely constrained by slow wall-clock convergence and high estimation variance.<n>We propose AdaLeZO, an Adaptive Layer-wise ZO optimization framework.
  arXiv  Detail & Related papers  (2026-04-20T13:37:31Z)
• Stability and Generalization of Push-Sum Based Decentralized Optimization over Directed Graphs [55.77845440440496]
  Push-based decentralized communication enables optimization over communication networks, where information exchange may be asymmetric.<n>We develop a unified uniform-stability framework for the Gradient Push (SGP) algorithm.<n>A key technical ingredient is an imbalance-aware generalization bound through two quantities.
  arXiv  Detail & Related papers  (2026-02-24T05:32:03Z)
• $γ$-weakly $θ$-up-concavity: Linearizable Non-Convex Optimization with Applications to DR-Submodular and OSS Functions [52.031993908548294]
  We introduce and $$-weakly $$-up-concavity, a novel first-orderizing condition that characterizes a broad approximation of such functions.<n>Our framework recovers the optimal coefficient for DR-submodular class and significantly improves existing approximation coefficients.
  arXiv  Detail & Related papers  (2026-02-13T22:34:44Z)
• Breaking the Stochasticity Barrier: An Adaptive Variance-Reduced Method for Variational Inequalities [0.0]
  We propose VR-A-A (VarianceReduced-Ascent with Armijo) as a novel algorithm for non-con optimization tasks.<n>We show that our method effectively suppresses limit cycles and accelerates convergence with reduced dependence on manual learning scheduling.
  arXiv  Detail & Related papers  (2026-01-30T14:43:07Z)
• Majorization-Minimization Networks for Inverse Problems: An Application to EEG Imaging [4.063392865490957]
  Inverse problems are often ill-posed and require optimization schemes with strong stability and convergence guarantees.<n>We propose a learned Majorization-Minimization (MM) framework for inverse problems within a bilevel optimization setting.<n>We learn a structured curvature majorant that governs each MM step while preserving classical MM descent guarantees.
  arXiv  Detail & Related papers  (2026-01-23T10:33:45Z)
• FedSVD: Adaptive Orthogonalization for Private Federated Learning with LoRA [68.44043212834204]
  Low-Rank Adaptation (LoRA) is widely used for efficient fine-tuning of language models in learning (FL)<n>Low-Rank Adaptation (LoRA) is widely used for efficient fine-tuning of language models in learning (FL)
  arXiv  Detail & Related papers  (2025-05-19T07:32:56Z)
• On PI Controllers for Updating Lagrange Multipliers in Constrained Optimization [16.40968330148623]
  This paper proposes the $nu$PI algorithm and contributes an optimization perspective on Lagrange multiplier updates based on PI controllers. We provide theoretical and empirical insights explaining the inability of momentum methods to address the shortcomings of gradient descent-ascent. We prove that $nu$PI generalizes popular momentum methods for single-objective minimization.
  arXiv  Detail & Related papers  (2024-06-07T00:13:31Z)
• Robust Online Control with Model Misspecification [96.23493624553998]
  We study online control of an unknown nonlinear dynamical system with model misspecification. Our study focuses on robustness, which measures how much deviation from the assumed linear approximation can be tolerated.
  arXiv  Detail & Related papers  (2021-07-16T07:04:35Z)
• Momentum Accelerates the Convergence of Stochastic AUPRC Maximization [80.8226518642952]
  We study optimization of areas under precision-recall curves (AUPRC), which is widely used for imbalanced tasks. We develop novel momentum methods with a better iteration of $O (1/epsilon4)$ for finding an $epsilon$stationary solution. We also design a novel family of adaptive methods with the same complexity of $O (1/epsilon4)$, which enjoy faster convergence in practice.
  arXiv  Detail & Related papers  (2021-07-02T16:21:52Z)
• Dynamics of Stochastic Momentum Methods on Large-scale, Quadratic Models [0.2741266294612776]
  We analyze a class of gradient algorithms with momentum on a high-dimensional random least squares problem. We show that (small-batch) momentum with a fixed momentum parameter provides no actual performance improvement over SGD when step sizes are adjusted correctly. In the non-strongly convex setting, it is possible to get a large improvement over SGD using momentum.
  arXiv  Detail & Related papers  (2021-06-07T15:08:24Z)
• Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to Improve Generalization [89.7882166459412]
  gradient noise (SGN) acts as implicit regularization for deep learning. Some works attempted to artificially simulate SGN by injecting random noise to improve deep learning. For simulating SGN at low computational costs and without changing the learning rate or batch size, we propose the Positive-Negative Momentum (PNM) approach.
  arXiv  Detail & Related papers  (2021-03-31T16:08:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.