Provable Edge-of-Stability for Adam on a One-Dimensional Quadratic

• URL: http://arxiv.org/abs/2608.20638v1
• Date: Fri, 21 Aug 2026 00:22:11 GMT
• Title: Provable Edge-of-Stability for Adam on a One-Dimensional Quadratic
• Abstract summary: We study uncorrected Adam on a one-dimensional quadratic, a clean setting where constant curvature isolates the edge-induced dynamics behind the EoS.<n>We identify settings in which this edge-seeking mechanism breaks down, including strictly subcritical periodic orbits and specially tuned trajectories that converge to the optimum.<n>These results give a concrete dynamical explanation for Adam's EoS in a setting free of evolving loss geometry, while also exposing its limitations.
• Score: 4.5845031477639155
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The edge-of-stability (EoS) phenomenon of Adam has been widely observed, while its underlying dynamical mechanism is not yet fully understood. We study uncorrected Adam on a one-dimensional quadratic, a clean setting where constant curvature isolates the optimizer-induced dynamics behind the EoS. We characterize the resulting dynamics across the parameter space. In broad regimes, we prove that Adam exhibits a restoring tendency toward its frozen stability threshold $2(1+β_1)/[η(1-β_1)]$. We also identify settings in which this edge-seeking mechanism breaks down, including strictly subcritical periodic orbits and specially tuned trajectories that converge to the optimum while remaining uniformly supercritical. These results give a concrete dynamical explanation for Adam's EoS in a setting free of evolving loss geometry, while also exposing its limitations.

Related papers

• Information Geometric Self-Organization at the Edge of Stability in High-Capacity Kernel Associative Memories [0.0]
  We show that optimal high-capacity memory representations are not formed in flat minima, but are sculpted at the highly curved boundaries of geometric singularities.<n>These findings suggest that optimal, high-capacity memory representations are not formed in flat minima, but are dynamically sculpted at the highly curved boundaries of geometric singularities.
  arXiv  Detail & Related papers  (2026-09-15T08:27:55Z)
• Geometric Collapse: When Vision Models Fail to Verify Physical Causality [52.43819179934414]
  We propose Scrambled Edges, a controlled counterfactual that injects salient edge-like cues.<n>With energy-matched and structure-matched controls, we isolate the effect of unsupported edge evidence from high-frequency energy and edge sparsity.<n>Across CNN/ViT/SSL depth predictors, Scrambled Edges induce up to 3.2x larger deviation from clean predictions than energy-matched noise.
  arXiv  Detail & Related papers  (2026-07-08T00:04:02Z)
• Disentangling Continuous-Time Latent Dynamics: Identifiability of Latent SDEs via Diffusion Shifts [60.95873409673775]
  identifiability in continuous-time latent differential equation models remains largely open.<n>We study additive-noise latent SDEs observed through an unknown nonlinear diffeomorphism.<n>We propose a two-stage estimator for latent disentanglement and optional graph recovery.
  arXiv  Detail & Related papers  (2026-06-26T16:18:28Z)
• Edge Flow: A Tractable and Predictive Continuous-Time Model for Gradient Descent at the Edge of Stability [11.598128347617957]
  We propose a tractable, faithful, and predictive model of gradient descent dynamics at the edge of stability (EoS)<n>We show that Edge Flow tracks the dynamics of gradient descent at least as faithfully as previously proposed continuous-time EoS models.
  arXiv  Detail & Related papers  (2026-06-16T15:45:47Z)
• High-Dimensional Latents Should Be Diagnosed Through Phase Structure [56.362776482614976]
  We study autoencoder and variational-autoencoder latent spaces through the lens of spin-glass theory.<n>We formalize a latent-space spin-glass dictionary for a fixed decoder.<n>We show that driving the latent system toward the edge-of-stability of the topological trivialization regime has concrete downstream consequences.
  arXiv  Detail & Related papers  (2026-05-23T06:17:23Z)
• A Rod Flow Model for Adam at the Edge of Stability [9.11122093402205]
  We extend rod flow to Adam by working in the joint phase space of parameters and first moment $(w, m).<n>We also develop rod flows for heavy ball momentum.<n>For all eight$s, we empirically evaluate rod flow on representative machine learning architectures.
  arXiv  Detail & Related papers  (2026-05-07T18:21:59Z)
• AdamO: A Collapse-Suppressed Optimizer for Offline RL [12.482937306811026]
  offline reinforcement learning can fail spectacularly when bootstrapped temporal-difference (TD) updates amplify their own errors, driving the critic toward extreme and unusable Q-values.<n>We model offline TD learning as a feedback system and analyze Adam-based critic updates.<n>We prove that this design theoretically guarantees worst-case task safety and preserves Adam's continuous-time dissipative dynamics.
  arXiv  Detail & Related papers  (2026-05-03T16:53:29Z)
• Towards Understanding Adam Convergence on Highly Degenerate Polynomials [12.224244942795695]
  We study the "natural" auto-convergence properties of Adam.<n>We find a class of highly degenerates where Adam converges automatically without additional schedulers.<n>We prove that Adam degenerates local linear convergence on these functions, significantly outperforming the sub- convergence of Gradient Descent and Momentum.
  arXiv  Detail & Related papers  (2026-03-10T12:30:20Z)
• GTS: Inference-Time Scaling of Latent Reasoning with a Learnable Gaussian Thought Sampler [54.10960908347221]
  We model latent thought exploration as conditional sampling from learnable densities and instantiate this idea as a Gaussian Thought Sampler (GTS)<n>GTS predicts context-dependent perturbation distributions over continuous reasoning states and is trained with GRPO-style policy optimization while keeping the backbone frozen.
  arXiv  Detail & Related papers  (2026-02-15T09:57:47Z)
• From Adam to Adam-Like Lagrangians: Second-Order Nonlocal Dynamics [0.0]
  We derive an accelerated continuous-time formulation of Adam by modeling it as a second-order integro-differential dynamical system.<n>We relate this inertial nonlocal model to an existing first-order nonlocal Adam flow through an $$-refinement limit, and we provide Lyapunov-based stability and convergence analyses.
  arXiv  Detail & Related papers  (2026-02-09T19:00:03Z)
• Implicit Bias of Gradient Descent for Logistic Regression at the Edge of Stability [69.01076284478151]
  In machine learning optimization, gradient descent (GD) often operates at the edge of stability (EoS) This paper studies the convergence and implicit bias of constant-stepsize GD for logistic regression on linearly separable data in the EoS regime.
  arXiv  Detail & Related papers  (2023-05-19T16:24:47Z)
• Beyond the Edge of Stability via Two-step Gradient Updates [49.03389279816152]
  Gradient Descent (GD) is a powerful workhorse of modern machine learning. GD's ability to find local minimisers is only guaranteed for losses with Lipschitz gradients. This work focuses on simple, yet representative, learning problems via analysis of two-step gradient updates.
  arXiv  Detail & Related papers  (2022-06-08T21:32:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.