When Do Autoregressive Sequence Models Forecast Physical Wavefields? A Controlled Study on Synthetic Seismograms

• URL: http://arxiv.org/abs/2606.10868v1
• Date: Tue, 09 Jun 2026 13:46:14 GMT
• Title: When Do Autoregressive Sequence Models Forecast Physical Wavefields? A Controlled Study on Synthetic Seismograms
• Abstract summary: Long-horizon autoregressive forecasting of physical signals is limited by error accumulation.<n>We ask when such rollout stays stable using synthetic three-component seismograms as a physically structured testbed.
• Score: 43.8784307709823
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Long-horizon autoregressive forecasting of oscillatory physical signals, such as seismograms, gravitational-wave strain, and similar wavefields is limited by error accumulation: as a causal model is fed its own outputs over hundreds of steps, small per-step errors compound into phase drift that pointwise metrics fail to detect. We ask when such rollout stays stable, using synthetic three-component seismograms as a physically structured testbed and the \textsc{SeismoGPT} autoregressive forecaster as the model under study. Through controlled, intra-architecture ablations evaluated on free-running rollout with paired significance tests, we isolate the contribution of each design choice. Multi-token prediction is the dominant stabilizer, accounting for almost the entire improvement over a single-token baseline ($+0.040$ median NCC); a horizon-embedding hybrid prediction head and a cross-horizon STFT-magnitude coherence loss each add a small but consistent further gain. Performance depends sharply on a context-ratio threshold near one, roughly the full P-S interval of observed signal, below which rollout generalization collapses. The dominant residual failure is a polarity inversion that a magnitude-based spectral loss cannot, by construction, penalize, identifying phase-aware objectives as the natural next step. We frame this as a controlled study of rollout stability on oscillatory wavefields, not a benchmark of forecasting architectures.

Related papers

• Orthogonal JEPA: Factorized Predictive States for Latent World Models [95.20158869896393]
  We introduce method, a latent world-modeling framework based on predictive factorization.<n>We show that method can allocate redundant capacity to dominant signals while providing weak or conflicting gradients to less dominant predictive structure.<n>Experiments on controlled vision, single-cell transcriptomics, longitudinal health records, continuous control, and molecular dynamics evaluate representation quality, forecasting, planning, and long-horizon stability.
  arXiv  Detail & Related papers  (2026-08-20T13:59:57Z)
• Link Prediction or Perdition: the Seeds of Instability in Knowledge Graph Embeddings [2.1174215880331766]
  Embedding models (KGEMs) constitute the main link prediction approach to complete knowledge graphs.<n>Standard evaluation protocols emphasize rank-based metrics such as MRR or Hits@$K$, but usually overlook the influence of random seeds on result stability.<n>We find that high-performance models actually produce divergent predictions at the triple level and highly variable spaces.
  arXiv  Detail & Related papers  (2026-06-02T09:13:50Z)
• Data-Driven Forecasting of three-Component Seismograms Using Transformer Architectures [43.8784307709823]
  We introduce textscSeismoGPT, a transformer-based autoregressive model designed to forecast three-component seismic waveforms directly in the time domain.<n> Evaluation is performed on synthetic seismograms spanning source depths of 5--100,km, epicentral distances of 10--90$circ$, and magnitudes $3 leq M_w leq 7$.
  arXiv  Detail & Related papers  (2026-06-01T21:38:03Z)
• Is Capability a Liability? More Capable Language Models Make Worse Forecasts When It Matters Most [11.4807141145402]
  In inverse scaling, more capable models produce worse distributional forecasts on forecasting problems with superlinear growth and tail risk of regime change.<n>The pattern appears on ForecastBench-Sim, in forecasting synthetic SIR epidemics with a matched linear control, and replicates in real-world datasets on COVID-19, measles, housing markets, and hyperinflation.<n>This inverse scaling does not appear on single-threshold metrics common in LLM forecasting benchmarks, reversing the sign of the capability--accuracy relationship on identical outputs.
  arXiv  Detail & Related papers  (2026-05-21T16:14:33Z)
• Improving Spatio-Temporal Residual Error Propagation by Mitigating Over-Squashing [4.62541204592513]
  We introduce Teger, a structured uncertainty module that overcomes the spa- tial and temporal limitations of error-related autoregressive forecasting.<n>Teger is backbone-agnostic, requiring only the latent state produced by any autoregressive encoder.
  arXiv  Detail & Related papers  (2026-05-18T08:51:33Z)
• PnP-Corrector: A Universal Correction Framework for Coupled Spatiotemporal Forecasting [62.22445401483844]
  Existing methods are severely constrained by the bottleneck of compounding errors.<n>We propose a universal framework called -Cast-Corrector (Plug-and-Play Corrector)<n>Our method significantly enhances the long-term stability of coupled forecasting systems.
  arXiv  Detail & Related papers  (2026-05-09T13:12:33Z)
• Towards Scaling Law Analysis For Spatiotemporal Weather Data [50.25336689856158]
  We extend neural scaling analysis for autoregressive weather forecasting from single-step training loss to long rollouts and per-channel metrics.<n>We find strong cross-channel and cross-horizon heterogeneity: pooled scaling can look favorable while many channels degrade at late leads.<n>We discuss implications for weighted objectives, horizon-aware curricula, and resource allocation across outputs.
  arXiv  Detail & Related papers  (2026-04-06T18:14:53Z)
• Gradients Must Earn Their Influence: Unifying SFT with Generalized Entropic Objectives [22.29000001610794]
  Standard negative log-likelihood for Supervised Fine-Tuning (SFT) applies uniform token-level weighting.<n>This rigidity creates a two-fold failure mode: (i) overemphasizing low-probability targets can amplify gradients on noisy supervision and disrupt robust priors, and (ii) uniform weighting provides weak sharpening when the model is already confident.<n>Existing methods fail to resolve the resulting plasticity--stability dilemma, often suppressing necessary learning signals alongside harmful ones.<n>We introduce Dynamic Entropy Fine-Tuning (DEFT), a parameter-free objective that modulates the
  arXiv  Detail & Related papers  (2026-02-11T22:56:43Z)
• Drift No More? Context Equilibria in Multi-Turn LLM Interactions [58.69551510148673]
  contexts drift is the gradual divergence of a model's outputs from goal-consistent behavior across turns.<n>Unlike single-turn errors, drift unfolds temporally and is poorly captured by static evaluation metrics.<n>We show that multi-turn drift can be understood as a controllable equilibrium phenomenon rather than as inevitable decay.
  arXiv  Detail & Related papers  (2025-10-09T04:48:49Z)
• Analytic Signal Phase in $N-D$ by Linear Symmetry Tensor--fingerprint modeling [69.35569554213679]
  We show that the Analytic Signal phase, and its gradient have a hitherto unstudied discontinuity in $2-D $ and higher dimensions. This shortcoming can result in severe artifacts whereas the problem does not exist in $1-D $ signals. We suggest the use of Linear Symmetry phase, relying on more than one set of Gabor filters, but with a negligible computational add-on.
  arXiv  Detail & Related papers  (2020-05-16T21:17:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.