PsiLogic: Chaos-Aware Active Cancellation for Adam with a Fair Cross-Domain Benchmark

• URL: http://arxiv.org/abs/2607.16268v1
• Date: Sun, 05 Jul 2026 14:56:54 GMT
• Title: PsiLogic: Chaos-Aware Active Cancellation for Adam with a Fair Cross-Domain Benchmark
• Abstract summary: PsiLogic stabilizes Adam with a dynamic Active Cancellation Term gated by a dual exponential moving average.<n>We evaluate PsiLogic against Adam, AdamW, and Lion using FairBench.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Adaptive optimizers such as Adam and AdamW apply the same update rule regardless of whether training is in a chaotic early phase or near convergence. We introduce PsiLogic, an optimizer that augments Adam with a dynamic Active Cancellation Term gated by a dual exponential moving average (EMA) of scale-normalized gradient norms. The resulting chaos detector strengthens damping when gradient statistics are unstable and fades to zero as training stabilizes, providing an implicit warmup without a hand-tuned schedule. We evaluate PsiLogic against Adam, AdamW, and Lion using FairBench -- a reproducible benchmark protocol with per-optimizer learning-rate sweeps, identical initialization per seed, and Welch t-tests. On an NVIDIA H100 80GB reference run (4 arenas, 3 seeds, 2000 steps, bf16 AMP), PsiLogic achieves the best validation metric in three of four arenas: NLP perplexity 7.79 +/- 0.18 vs. 8.17 +/- 0.08 (AdamW, p = 0.049), ViT top-1 accuracy 0.244 +/- 0.006 vs. 0.223 +/- 0.002 (AdamW, p = 0.015), and ResNet top-1 accuracy 0.222 +/- 0.001 vs. 0.172 +/- 0.004 (Adam, p = 0.001). On diffusion, validation MSE is statistically tied with Adam/AdamW (p = 0.49). ResNet accuracy vs. AdamW is a numerical tie without significance at three seeds (p = 0.44). Peak GPU memory is comparable across optimizers; PsiLogic incurs 1.2--1.8x wall-clock overhead on transformer-heavy arenas (implementation-bound). We release an open-source PyTorch implementation, the full FairBench harness, and all raw CSV outputs to support independent verification.

Related papers

• Adaptive Runge-Kutta Step Control Buys Training Loss, Not Generalization: An Honest Compute-Matched Study of RK-Adam Optimizers [0.0]
  We build a representative Adam variant (Bogacki-Shampine 3(2) RK pair, FSAL reuse, local-error step control)<n>We evaluate it under a strict compute-matched protocol giving every method the same gradient-evaluation budget.
  arXiv  Detail & Related papers  (2026-07-16T03:12:58Z)
• LLM Features Can Hurt GNNs: Concatenation Interference on Homophilous Graph Benchmarks [10.283803346813047]
  Adding LLM-generated node features to graph neural networks (GNNs) is widely reported to improve accuracy on standard benchmarks.<n>We document a contrasting observation: when LLM features are introduced through pure input concatenation, they can systematically degrade accuracy on homophilous benchmarks where end-to-end pipelines succeed.
  arXiv  Detail & Related papers  (2026-06-16T06:30:30Z)
• THEIA: Learning Complete Kleene Three-Valued Logic in a Pure-Neural Modular Architecture [0.0]
  THEIA is a 2.75M modular neural architecture that learns the complete Kleene three-valued logic (K3) truth table from task data without external symbolic inference or hand-encoded K3 gate primitives.<n> Transformer baselines also reach >99% on all 39 rules, and flats match THEIA on Phase-1 accuracy within 0.04pp.
  arXiv  Detail & Related papers  (2026-04-13T10:44:15Z)
• Combining Adam and its Inverse Counterpart to Enhance Generalization of Deep Learning Optimizers [57.049014152026864]
  In the training of neural networks, adaptive moment estimation (Adam) typically converges fast but exhibits suboptimal generalization performance.<n>To enhance its ability to find flat minima, we propose its new variant named inverse Adam (InvAdam)<n>InvAdam computes element-wise multiplication of the first-order and second-order moments, while Adam computes the element-wise division of these two moments.
  arXiv  Detail & Related papers  (2026-03-07T09:15:30Z)
• Fast Compute for ML Optimization [0.0]
  We study optimization for losses that admit a variance-mean scale-mixture representation.<n>The resulting Scale Mixture EM (SM-EM) algorithm removes user-specified learning-rate and momentum schedules.<n>For the base (non-accelerated) algorithm, EM monotonicity guarantees nonincreasing objective values; adding Nesterov extrapolation trades this guarantee for faster empirical convergence.
  arXiv  Detail & Related papers  (2026-02-15T19:09:58Z)
• Kourkoutas-Beta: A Sunspike-Driven Adam Optimizer with Desert Flair [0.0]
  Transformer neural networks are increasingly used for physics-based problems.<n>In data-driven PDE surrogates, training samples from varying boundary and initial conditions can cause erratic losses and spiky gradients.<n>We introduce Kourkoutas-Beta, an Adam-style discount where the fixed second-moment beta2 is replaced by a layer-wise dynamic value.
  arXiv  Detail & Related papers  (2025-08-18T15:16:54Z)
• Maximizing Communication Efficiency for Large-scale Training via 0/1 Adam [49.426602335460295]
  1-bit communication is an effective method to scale up model training, and has been studied extensively on SGD. We propose 0/1 Adam, which improves upon the state-of-the-art 1-bit Adam via two novel methods.
  arXiv  Detail & Related papers  (2022-02-12T08:02:23Z)
• High performing ensemble of convolutional neural networks for insect pest image detection [124.23179560022761]
  Pest infestation is a major cause of crop damage and lost revenues worldwide. We generate ensembles of CNNs based on different topologies. Two new Adam algorithms for deep network optimization are proposed.
  arXiv  Detail & Related papers  (2021-08-28T00:49:11Z)
• Adam$^+$: A Stochastic Method with Adaptive Variance Reduction [56.051001950733315]
  Adam is a widely used optimization method for deep learning applications. We propose a new method named Adam$+$ (pronounced as Adam-plus) Our empirical studies on various deep learning tasks, including image classification, language modeling, and automatic speech recognition, demonstrate that Adam$+$ significantly outperforms Adam.
  arXiv  Detail & Related papers  (2020-11-24T09:28:53Z)
• ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning [91.13797346047984]
  We introduce ADAHESSIAN, a second order optimization algorithm which dynamically incorporates the curvature of the loss function via ADAptive estimates. We show that ADAHESSIAN achieves new state-of-the-art results by a large margin as compared to other adaptive optimization methods.
  arXiv  Detail & Related papers  (2020-06-01T05:00:51Z)
• A Simple Convergence Proof of Adam and Adagrad [74.24716715922759]
  We show a proof of convergence between the Adam Adagrad and $O(d(N)/st)$ algorithms. Adam converges with the same convergence $O(d(N)/st)$ when used with the default parameters.
  arXiv  Detail & Related papers  (2020-03-05T01:56:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.