StoSignSGD: Unbiased Structural Stochasticity Fixes SignSGD for Training Large Language Models

• URL: http://arxiv.org/abs/2604.15416v1
• Date: Thu, 16 Apr 2026 17:55:36 GMT
• Title: StoSignSGD: Unbiased Structural Stochasticity Fixes SignSGD for Training Large Language Models
• Abstract summary: Sign-based optimization algorithms, such as SignSGD, have garnered significant attention for their remarkable performance in distributed learning and training large foundation models.<n>Despite their empirical superiority, SignSGD is known to diverge on non-text-smooth objectives.<n>We propose bfStoSignSGD, an algorithm that injects structurality into the sign operator while maintaining an unbiased update step.
• Score: 16.690425653502256
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sign-based optimization algorithms, such as SignSGD, have garnered significant attention for their remarkable performance in distributed learning and training large foundation models. Despite their empirical superiority, SignSGD is known to diverge on non-smooth objectives, which are ubiquitous in modern machine learning due to ReLUs, max-pools, and mixture-of-experts. To overcome this fundamental limitation, we propose \textbf{StoSignSGD}, an algorithm that injects structural stochasticity into the sign operator while maintaining an unbiased update step. In the regime of (online) convex optimization, our theoretical analysis shows that StoSignSGD rigorously resolves the non-convergence issues of SignSGD, achieving a sharp convergence rate matching the lower bound. For the more challenging non-convex non-smooth optimization, we introduce generalized stationary measures that encompass prior definitions, proving that StoSignSGD improves upon the best-known complexity bounds by dimensional factors. Empirically, StoSignSGD exhibits robust stability and superior efficiency across diverse large language model (LLM) training regimes. Notably, in low-precision FP8 pretraining -- a setting where AdamW fails catastrophically -- StoSignSGD remains highly stable and yields a remarkable 1.44$\times$ to 2.14$\times$ speedup relative to established baselines. Furthermore, when fine-tuning 7B LLMs on mathematical reasoning tasks, StoSignSGD delivers substantial performance gains over both AdamW and SignSGD. Finally, to dissect the mechanisms driving its success, we develop a sign conversion framework capable of transforming any general optimizer into its unbiased, sign-based counterpart. Utilizing this framework, we deconstruct the core components of StoSignSGD and present a comprehensive ablation study to empirically validate our algorithmic design choices.

Related papers

• A Critical Synthesis of Uncertainty Quantification and Foundation Models for Semantic Segmentation [8.144213556496569]
  Foundation models are breaking what seemed to be impossible not long ago by enabling unprecedented accuracy and cross-domain generalization.<n>Their lack of interpretability, tendency to be overconfident, and sensitivity to real-world domain shifts pose critical challenges for safety- and mission-critical applications.<n>Uncertainty quantification (UQ) offers a principled way to address these issues, but its integration into segmentation foundation models has yet to be explored.
  arXiv  Detail & Related papers  (2026-08-19T09:09:42Z)
• Factor Augmented High-Dimensional SGD [4.542148087324063]
  gradient descent (SGD) is a fundamental optimization algorithm widely used in modern machine learning.<n>We propose Factor-Augmented SGD (FSGD), a new optimization method that leverages latent factor representations in high-dimensional learning tasks.
  arXiv  Detail & Related papers  (2026-05-19T03:10:33Z)
• When and Why SignSGD Outperforms SGD: A Theoretical Study Based on $\ell_1$-norm Lower Bounds [15.724495678714007]
  Sign-based optimization algorithms, such as SignSGD and Muon, have garnered significant attention for their remarkable performance in training large foundation models.<n>We show that SignSGD effectively reduces the complexity by a factor of $d$ under emphsparse noise, where $d$ is the problem dimension.
  arXiv  Detail & Related papers  (2026-05-07T17:32:09Z)
• Mitigating Error Amplification in Fast Adversarial Training [58.74042726356826]
  Fast Adversarial Training (FAT) has proven effective in enhancing model robustness by encouraging networks to learn perturbation-invariant representations.<n>FAT often suffers from catastrophic overfitting (CO), where the model overfits to the training attack and fails to generalize to unseen ones.<n>We propose a Distribution-aware Dynamic Guidance (DDG) strategy that dynamically adjusts both the perturbation budget and supervision signal.
  arXiv  Detail & Related papers  (2026-04-27T11:23:18Z)
• Think Twice Before You Write -- an Entropy-based Decoding Strategy to Enhance LLM Reasoning [32.332197731504046]
  Decoding strategies play a central role in shaping the reasoning ability of large language models.<n>Traditional methods such as greedy decoding and beam search often suffer from error propagation.<n>We propose an entropy-guided decoding framework that introduces token-level adaptivity into generation.
  arXiv  Detail & Related papers  (2026-03-10T23:08:26Z)
• The Power of Decaying Steps: Enhancing Attack Stability and Transferability for Sign-based Optimizers [9.020853139493239]
  We argue that one cause for non-convergence and instability is their non-decaying step-size scheduling.<n>We propose a series of new attack algorithms that enforce Monotonically Decreasing Coordinate-wise Step-sizes within sign-based adversarials.
  arXiv  Detail & Related papers  (2026-02-22T08:37:06Z)
• Principled RL for Diffusion LLMs Emerges from a Sequence-Level Perspective [85.06838178922791]
  Reinforcement Learning (RL) has proven highly effective for autoregressive language models.<n>But adapting these methods to diffusion large language models (dLLMs) presents fundamental challenges.<n>We propose a principled RL framework that treats entire sequence generation as a single action and uses the ELBO as a tractable sequence-level likelihood proxy.
  arXiv  Detail & Related papers  (2025-12-03T13:05:32Z)
• Sycophancy Mitigation Through Reinforcement Learning with Uncertainty-Aware Adaptive Reasoning Trajectories [58.988535279557546]
  We introduce textbf sycophancy Mitigation through Adaptive Reasoning Trajectories.<n>We show that SMART significantly reduces sycophantic behavior while preserving strong performance on out-of-distribution inputs.
  arXiv  Detail & Related papers  (2025-09-20T17:09:14Z)
• NDCG-Consistent Softmax Approximation with Accelerated Convergence [67.10365329542365]
  We propose novel loss formulations that align directly with ranking metrics.<n>We integrate the proposed RG losses with the highly efficient Alternating Least Squares (ALS) optimization method.<n> Empirical evaluations on real-world datasets demonstrate that our approach achieves comparable or superior ranking performance.
  arXiv  Detail & Related papers  (2025-06-11T06:59:17Z)
• Leveraging Coordinate Momentum in SignSGD and Muon: Memory-Optimized Zero-Order [39.25335214877435]
  Fine-tuning Large Language Models (LLMs) is essential for adapting pre-trained models to downstream tasks.<n>Traditional first-order algorithms incur prohibitive memory and computational costs that scale poorly with model size.<n>We propose zero-order (ZO) optimization methods as a memory- and compute-efficient alternative.
  arXiv  Detail & Related papers  (2025-06-04T20:27:17Z)
• Benign Underfitting of Stochastic Gradient Descent [72.38051710389732]
  We study to what extent may gradient descent (SGD) be understood as a "conventional" learning rule that achieves generalization performance by obtaining a good fit training data. We analyze the closely related with-replacement SGD, for which an analogous phenomenon does not occur and prove that its population risk does in fact converge at the optimal rate.
  arXiv  Detail & Related papers  (2022-02-27T13:25:01Z)
• Revisiting and Advancing Fast Adversarial Training Through The Lens of Bi-Level Optimization [60.72410937614299]
  We propose a new tractable bi-level optimization problem, design and analyze a new set of algorithms termed Bi-level AT (FAST-BAT) FAST-BAT is capable of defending sign-based projected descent (PGD) attacks without calling any gradient sign method and explicit robust regularization.
  arXiv  Detail & Related papers  (2021-12-23T06:25:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.