Transformer Learning of Chaotic Collective Dynamics in Many-Body Systems

• URL: http://arxiv.org/abs/2601.19080v1
• Date: Tue, 27 Jan 2026 01:33:33 GMT
• Title: Transformer Learning of Chaotic Collective Dynamics in Many-Body Systems
• Authors: Ho Jang, Gia-Wei Chern,
• Abstract summary: We show that a self-attention-based transformer framework provides an effective approach for modeling chaotic collective dynamics.<n>We study the one-dimensional semiclassical Holstein model, where interaction quenches induce strongly nonlinear and chaotic dynamics.<n>Our results establish self-attention as a powerful mechanism for learning effective reduced dynamics in chaotic many-body systems.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning reduced descriptions of chaotic many-body dynamics is fundamentally challenging: although microscopic equations are Markovian, collective observables exhibit strong memory and exponential sensitivity to initial conditions and prediction errors. We show that a self-attention-based transformer framework provides an effective approach for modeling such chaotic collective dynamics directly from time-series data. By selectively reweighting long-range temporal correlations, the transformer learns a non-Markovian reduced description that overcomes intrinsic limitations of conventional recurrent architectures. As a concrete demonstration, we study the one-dimensional semiclassical Holstein model, where interaction quenches induce strongly nonlinear and chaotic dynamics of the charge-density-wave order parameter. While pointwise predictions inevitably diverge at long times, the transformer faithfully reproduces the statistical "climate" of the chaos, including temporal correlations and characteristic decay scales. Our results establish self-attention as a powerful mechanism for learning effective reduced dynamics in chaotic many-body systems.

Related papers

• KoopGen: Koopman Generator Networks for Representing and Predicting Dynamical Systems with Continuous Spectra [65.11254608352982]
  We introduce a generator-based neural Koopman framework that models dynamics through a structured, state-dependent representation of Koopman generators.<n>By exploiting the intrinsic Cartesian decomposition into skew-adjoint and self-adjoint components, KoopGen separates conservative transport from irreversible dissipation.
  arXiv  Detail & Related papers  (2026-02-15T06:32:23Z)
• Dynamical Systems Analysis Reveals Functional Regimes in Large Language Models [0.8694591156258423]
  Large language models perform text generation through high-dimensional internal dynamics.<n>Most interpretability approaches emphasise static representations or causal interventions, leaving temporal structure largely unexplored.<n>We discuss a composite dynamical metric, computed from activation time-series during autoregressive generation.
  arXiv  Detail & Related papers  (2026-01-11T21:57:52Z)
• A Mechanistic Analysis of Transformers for Dynamical Systems [4.590170084532207]
  We study the representational capabilities and limitations of single-layer Transformers when applied to dynamical data.<n>For linear systems, we show that the convexity constraint imposed by softmax attention fundamentally restricts the class of dynamics that can be represented.<n>For nonlinear systems under partial observability, attention instead acts as an adaptive delay-embedding mechanism.
  arXiv  Detail & Related papers  (2025-12-24T11:21:07Z)
• UnCLe: Towards Scalable Dynamic Causal Discovery in Non-linear Temporal Systems [4.9593603893289115]
  We propose UnCLe, a novel deep learning method for scalable dynamic causal discovery.<n>UnCLe employs a pair of Uncoupler and Recoupler networks to disentangle input time series into semantic representations.<n>It estimates dynamic causal influences by analyzing datapoint-wise prediction errors induced by temporal perturbations.
  arXiv  Detail & Related papers  (2025-11-05T04:34:31Z)
• Drift No More? Context Equilibria in Multi-Turn LLM Interactions [58.69551510148673]
  contexts drift is the gradual divergence of a model's outputs from goal-consistent behavior across turns.<n>Unlike single-turn errors, drift unfolds temporally and is poorly captured by static evaluation metrics.<n>We show that multi-turn drift can be understood as a controllable equilibrium phenomenon rather than as inevitable decay.
  arXiv  Detail & Related papers  (2025-10-09T04:48:49Z)
• Data-driven particle dynamics: Structure-preserving coarse-graining for emergent behavior in non-equilibrium systems [0.8796261172196743]
  Multiscale systems are notoriously challenging to simulate as shorttemporal scales must be appropriately linked to emergent bulk physics.<n>We propose a framework using the metriplectic bracket formalism that preserves discrete notions of the first and second laws of thermodynamics.<n>We provide open-source implementations in both PyTorch and LAMMPS, enabling large-scale inference and rearrangement to diverse particle-based systems.
  arXiv  Detail & Related papers  (2025-08-18T02:10:18Z)
• Langevin Flows for Modeling Neural Latent Dynamics [81.81271685018284]
  We introduce LangevinFlow, a sequential Variational Auto-Encoder where the time evolution of latent variables is governed by the underdamped Langevin equation.<n>Our approach incorporates physical priors -- such as inertia, damping, a learned potential function, and forces -- to represent both autonomous and non-autonomous processes in neural systems.<n>Our method outperforms state-of-the-art baselines on synthetic neural populations generated by a Lorenz attractor.
  arXiv  Detail & Related papers  (2025-07-15T17:57:48Z)
• Scaling Collapse Reveals Universal Dynamics in Compute-Optimally Trained Neural Networks [59.552873049024775]
  We show that compute-optimally trained models exhibit a remarkably precise universality.<n>With learning rate decay, the collapse becomes so tight that differences in the normalized curves across models fall below the noise floor.<n>We explain these phenomena by connecting collapse to the power-law structure in typical neural scaling laws.
  arXiv  Detail & Related papers  (2025-07-02T20:03:34Z)
• StFT: Spatio-temporal Fourier Transformer for Long-term Dynamics Prediction [10.64762092324374]
  We propose an autoregressive Spatio-temporal Transformer (FTStours) to learn the system dynamics at a distinct scale.<n>FTStours captures the underlying dynamics across both macro- and micro- spatial scales.<n> Evaluations conducted on three benchmark datasets demonstrate the advantages of our approach over state-of-the-art ML methods.
  arXiv  Detail & Related papers  (2025-03-14T22:04:03Z)
• Neural Persistence Dynamics [8.197801260302642]
  We consider the problem of learning the dynamics in the topology of time-evolving point clouds. Our proposed model - $textitNeural Persistence Dynamics$ - substantially outperforms the state-of-the-art across a diverse set of parameter regression tasks.
  arXiv  Detail & Related papers  (2024-05-24T17:20:18Z)
• Attractor Memory for Long-Term Time Series Forecasting: A Chaos Perspective [63.60312929416228]
  textbftextitAttraos incorporates chaos theory into long-term time series forecasting. We show that Attraos outperforms various LTSF methods on mainstream datasets and chaotic datasets with only one-twelfth of the parameters compared to PatchTST.
  arXiv  Detail & Related papers  (2024-02-18T05:35:01Z)
• Dynamics with autoregressive neural quantum states: application to critical quench dynamics [41.94295877935867]
  We present an alternative general scheme that enables one to capture long-time dynamics of quantum systems in a stable fashion. We apply the scheme to time-dependent quench dynamics by investigating the Kibble-Zurek mechanism in the two-dimensional quantum Ising model.
  arXiv  Detail & Related papers  (2022-09-07T15:50:00Z)
• Stochastically forced ensemble dynamic mode decomposition for forecasting and analysis of near-periodic systems [65.44033635330604]
  We introduce a novel load forecasting method in which observed dynamics are modeled as a forced linear system. We show that its use of intrinsic linear dynamics offers a number of desirable properties in terms of interpretability and parsimony. Results are presented for a test case using load data from an electrical grid.
  arXiv  Detail & Related papers  (2020-10-08T20:25:52Z)
• Bistability and time crystals in long-ranged directed percolation [0.0]
  We propose a simple cellular automaton with power-law interactions that gives rise to a bistable phase of long-ranged directed percolation. Our work thus provides a firm example of a classical discrete time crystal phase of matter.
  arXiv  Detail & Related papers  (2020-04-27T18:00:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.