Dynamical phase selection controls compute scaling in looped transformers

• URL: http://arxiv.org/abs/2608.26556v1
• Date: Thu, 27 Aug 2026 02:54:02 GMT
• Title: Dynamical phase selection controls compute scaling in looped transformers
• Abstract summary: We show that networks with identical architecture and objective, trained to identical accuracy, nevertheless realize distinct dynamical phases.<n>The phases are distinguished by their bifurcation mechanisms, including a saddle-node fold and a Neimark-Sacker-type transition to bounded nonstationary motion.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A looped transformer performs inference by iterating a weight-tied map, making its computation a dynamical process whose cost is set by the resulting inference dynamics. Here we show that networks with identical architecture and objective, trained to identical accuracy, nevertheless realize distinct dynamical phases depending strongly on initialization, and that the bifurcation defining each phase determines how test-time compute scales. The phases are distinguished by their bifurcation mechanisms, including a saddle-node fold and a Neimark-Sacker-type transition to bounded nonstationary motion. In the fold phase, a one-dimensional normal-form reduction predicts both the relaxation-time and spectral-gap amplitudes from local derivatives of the trained map, yielding the parameter-free relation $τ(\varepsilon)[1-λ_{\max}(-\varepsilon)]\toπ$. Composed with a regular distribution of problem difficulty, the same critical slowing down produces the workload-level tail $P(τ>N)\sim N^{-2}$. In the Neimark--Sacker phase, the fold scaling law disappears rather than merely changing its prefactor. Thus, test-time compute is not determined by architecture alone. It is governed by the dynamical phase of the solution found by training.

Related papers

• The Geometric Phase as a Diagnostic for Driven-Dissipative Oscillators [13.38226745175549]
  We show that driven-dissipative quantum oscillators lock their phase, deform their limit cycles, and undergo dissipative phase transitions.<n>This behavior is read from unrelated quantities defined on the same steady-state density matrix.<n>We show that a single geometric quantity organizes them.
  arXiv  Detail & Related papers  (2026-08-26T12:52:36Z)
• Stabilizing Extrapolation in Looped Transformers via Learned Stochastic Stopping [56.14767235650558]
  Looped Transformers, which repeatedly apply a shared transformer block, are an architecturally natural fit for variable-length computation tasks.<n>We trace this variance to the spurious correlation in simple algorithmic tasks between sequence length and number of loops.<n>Our work suggests that "when to stop" should be treated as a training design choice, not merely an inference-time-allocation rule.
  arXiv  Detail & Related papers  (2026-06-29T08:58:09Z)
• On the Mechanism and Dynamics of Modular Addition: Fourier Features, Lottery Ticket, and Grokking [49.1352577985191]
  We present a comprehensive analysis of how two-layer neural networks learn features to solve the modular addition task.<n>Our work provides a full mechanistic interpretation of the learned model and a theoretical explanation of its training dynamics.
  arXiv  Detail & Related papers  (2026-02-18T20:25:13Z)
• Latent Object Permanence: Topological Phase Transitions, Free-Energy Principles, and Renormalization Group Flows in Deep Transformer Manifolds [0.5729426778193398]
  We study the emergence of multi-step reasoning in deep Transformer language models through a geometric and statistical-physics lens.<n>We formalize the forward pass as a discrete coarse-graining map and relate the appearance of stable "concept basins" to fixed points of this renormalization-like dynamics.<n>The resulting low-entropy regime is characterized by a spectral tail collapse and by the formation of transient, reusable object-like structures in representation space.
  arXiv  Detail & Related papers  (2026-01-16T23:11:02Z)
• How Transformers Learn Regular Language Recognition: A Theoretical Study on Training Dynamics and Implicit Bias [48.9399496805422]
  We focus on two representative tasks in the category of regular language recognition, known as even pairs' and parity check'<n>Our goal is to explore how a one-layer transformer, consisting of an attention layer followed by a linear layer, learns to solve these tasks.
  arXiv  Detail & Related papers  (2025-05-02T00:07:35Z)
• Dynamical transitions from slow to fast relaxation in random open quantum systems [0.0]
  We study a model in which the system Hamiltonian and its couplings to the noise are random matrices whose entries decay as power laws of distance. The steady state is always featureless, but the rate at which it is approached exhibits three phases depending on $alpha_H$ and $alpha_L$. Within perturbation theory, the phase boundaries in the $(alpha_H, alpha_L)$ plane differ for weak and strong dissipation, suggesting phase transitions as a function of noise strength.
  arXiv  Detail & Related papers  (2022-11-23T20:56:46Z)
• Superdiffusion in random two dimensional system with time-reversal symmetry and long-range hopping [45.873301228345696]
  localization problem in the crossover regime for the dimension $d=2$ and hopping $V(r) propto r-2$ is not resolved yet. We show that for the hopping determined by two-dimensional anisotropic dipole-dipole interactions there exist two distinguishable phases at weak and strong disorder.
  arXiv  Detail & Related papers  (2022-05-29T16:53:20Z)
• On optimization of coherent and incoherent controls for two-level quantum systems [77.34726150561087]
  This article considers some control problems for closed and open two-level quantum systems. The closed system's dynamics is governed by the Schr"odinger equation with coherent control. The open system's dynamics is governed by the Gorini-Kossakowski-Sudarshan-Lindblad master equation.
  arXiv  Detail & Related papers  (2022-05-05T09:08:03Z)
• Spectral Analysis of Product Formulas for Quantum Simulation [0.0]
  We show that the Trotter step size needed to estimate an energy eigenvalue within precision can be improved in scaling from $epsilon$ to $epsilon1/2$ for a large class of systems. Results partially generalize to diabatic processes, which remain in a narrow energy band separated from the rest of the spectrum by a gap.
  arXiv  Detail & Related papers  (2021-02-25T03:17:25Z)
• Analytic Signal Phase in $N-D$ by Linear Symmetry Tensor--fingerprint modeling [69.35569554213679]
  We show that the Analytic Signal phase, and its gradient have a hitherto unstudied discontinuity in $2-D $ and higher dimensions. This shortcoming can result in severe artifacts whereas the problem does not exist in $1-D $ signals. We suggest the use of Linear Symmetry phase, relying on more than one set of Gabor filters, but with a negligible computational add-on.
  arXiv  Detail & Related papers  (2020-05-16T21:17:26Z)
• Discrete truncated Wigner approach to dynamical phase transitions in Ising models after a quantum quench [0.0]
  We study dynamical phase transitions arising in the steady state of transverse-field Ising models after a quantum quench. We find identical exponents for $alpha lesssim 0.5$, suggesting that the dynamical transitions in this regime fall into the same universality class as the nonergodic mean-field limit.
  arXiv  Detail & Related papers  (2020-04-21T08:20:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.