StFT: Spatio-temporal Fourier Transformer for Long-term Dynamics Prediction

• URL: http://arxiv.org/abs/2503.11899v2
• Date: Thu, 18 Sep 2025 22:41:01 GMT
• Title: StFT: Spatio-temporal Fourier Transformer for Long-term Dynamics Prediction
• Authors: Da Long, Shandian Zhe, Samuel Williams, Leonid Oliker, Zhe Bai,
• Abstract summary: We propose an autoregressive Spatio-temporal Transformer (FTStours) to learn the system dynamics at a distinct scale.<n>FTStours captures the underlying dynamics across both macro- and micro- spatial scales.<n> Evaluations conducted on three benchmark datasets demonstrate the advantages of our approach over state-of-the-art ML methods.
• Score: 10.64762092324374
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Simulating the long-term dynamics of multi-scale and multi-physics systems poses a significant challenge in understanding complex phenomena across science and engineering. The complexity arises from the intricate interactions between scales and the interplay of diverse physical processes, which manifest in PDEs through coupled, nonlinear terms that govern the evolution of multiple physical fields across scales. Neural operators have shown potential in short-term prediction of such complex spatio-temporal dynamics; however, achieving stable high-fidelity predictions and providing robust uncertainty quantification over extended time horizons remains an open and unsolved area of research. These limitations often lead to stability degradation with rapid error accumulation, particularly in long-term forecasting of systems characterized by multi-scale behaviors involving dynamics of different orders. To address these challenges, we propose an autoregressive Spatio-temporal Fourier Transformer (StFT), in which each transformer block is designed to learn the system dynamics at a distinct scale through a dual-path architecture that integrates frequency-domain and spatio-temporal representations. By leveraging a structured hierarchy of \ours blocks, the resulting model explicitly captures the underlying dynamics across both macro- and micro- spatial scales. Furthermore, a generative residual correction mechanism is introduced to learn a probabilistic refinement temporally while simultaneously quantifying prediction uncertainties, enhancing both the accuracy and reliability of long-term probabilistic forecasting. Evaluations conducted on three benchmark datasets (plasma, fluid, and atmospheric dynamics) demonstrate the advantages of our approach over state-of-the-art ML methods.

Related papers

• KoopGen: Koopman Generator Networks for Representing and Predicting Dynamical Systems with Continuous Spectra [65.11254608352982]
  We introduce a generator-based neural Koopman framework that models dynamics through a structured, state-dependent representation of Koopman generators.<n>By exploiting the intrinsic Cartesian decomposition into skew-adjoint and self-adjoint components, KoopGen separates conservative transport from irreversible dissipation.
  arXiv  Detail & Related papers  (2026-02-15T06:32:23Z)
• RainDiff: End-to-end Precipitation Nowcasting Via Token-wise Attention Diffusion [64.49056527678606]
  We propose a Token-wise Attention integrated into not only the U-Net diffusion model but also the radar-temporal encoder.<n>Unlike prior approaches, our method integrates attention into the architecture without incurring the high resource cost typical of pixel-space diffusion.<n>Our experiments and evaluations demonstrate that the proposed method significantly outperforms state-of-the-art approaches, robustness local fidelity, generalization, and superior in complex precipitation forecasting scenarios.
  arXiv  Detail & Related papers  (2025-10-16T17:59:13Z)
• Forecasting Low-Dimensional Turbulence via Multi-Dimensional Hybrid Quantum Reservoir Computing [0.0]
  We introduce a hybrid quantum-classical reservoir architecture capable of handling multivariate time series through quantum evolution combined with classical memory enhancement.<n>We apply this framework to two paradigmatic models of chaotic behavior in fluid dynamics, where multiscale dynamics and nonlinearities play a dominant role.<n>The robustness observed and reliable performances for both dynamical systems suggest that this hybrid quantum approach offers a flexible platform for modelling complex nonlinear time series.
  arXiv  Detail & Related papers  (2025-09-04T08:37:48Z)
• Langevin Flows for Modeling Neural Latent Dynamics [81.81271685018284]
  We introduce LangevinFlow, a sequential Variational Auto-Encoder where the time evolution of latent variables is governed by the underdamped Langevin equation.<n>Our approach incorporates physical priors -- such as inertia, damping, a learned potential function, and forces -- to represent both autonomous and non-autonomous processes in neural systems.<n>Our method outperforms state-of-the-art baselines on synthetic neural populations generated by a Lorenz attractor.
  arXiv  Detail & Related papers  (2025-07-15T17:57:48Z)
• A Generalizable Physics-Enhanced State Space Model for Long-Term Dynamics Forecasting in Complex Environments [16.251382658415494]
  We propose Phy-SSM, a generalizable method that integrates partial physics knowledge into state space models.<n>Our motivation is that SSMs can effectively capture long-range dependencies in sequential data and model continuous dynamical systems.<n>Experiments on three real-world applications, including vehicle motion prediction, drone state prediction, and COVID-19 epidemiology forecasting, demonstrate the superior performance of Phy-SSM.
  arXiv  Detail & Related papers  (2025-07-14T20:46:12Z)
• Delayformer: spatiotemporal transformation for predicting high-dimensional dynamics [4.984442247236504]
  Delayformer is a framework for simultaneously predicting dynamics of all variables.<n>From dynamical systems viewpoint, Delayformer predicts system states rather than individual variables.<n> Delayformer outperforms current state-of-the-art methods in forecasting tasks on both synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2025-06-13T07:30:12Z)
• Hierarchical Implicit Neural Emulators [20.09615751270837]
  We introduce a multiscale implicit neural emulator that enhances long-term prediction accuracy.<n>Our approach leverages predictions several steps ahead in time at increasing compression rates for next-timestep refinements.<n> Experiments on turbulent fluid dynamics show that our method achieves high short-term accuracy and produces long-term stable forecasts.
  arXiv  Detail & Related papers  (2025-06-05T00:28:26Z)
• Mamba Integrated with Physics Principles Masters Long-term Chaotic System Forecasting [26.45571254488745]
  PhyxMamba is a framework that integrates a Mamba-based state-space model with physics-informed principles to forecast long-term behavior of chaotic systems.<n>We show that PhyxMamba delivers superior forecasting accuracy and faithfully captures essential statistics from short-term historical observations.
  arXiv  Detail & Related papers  (2025-05-29T08:56:45Z)
• Learning and Current Prediction of PMSM Drive via Differential Neural Networks [13.370017978792479]
  This study presents a novel approach utilizing differential neural networks (DNNs) to model nonlinear systems.<n>The efficacy of our approach is validated through experiments conducted under various load disturbances and no-load conditions.
  arXiv  Detail & Related papers  (2024-12-12T07:43:27Z)
• Ab-Initio Approach to Many-Body Quantum Spin Dynamics [0.0]
  We use the multilayer multiconfiguration time-dependent Hartree framework to simulate the many-body spin dynamics of the Heisenberg model.<n> Benchmarks with analytical and exact numerical approaches show that ML-MCTDH accurately captures the time evolution of one- and two-body observables.<n>The behavior of the corresponding entanglement dynamics is analyzed to reveal the complexity of the quantum states.
  arXiv  Detail & Related papers  (2024-11-20T10:42:35Z)
• Dynamical Mean-Field Theory of Self-Attention Neural Networks [0.0]
  Transformer-based models have demonstrated exceptional performance across diverse domains. Little is known about how they operate or what are their expected dynamics. We use methods for the study of asymmetric Hopfield networks in nonequilibrium regimes.
  arXiv  Detail & Related papers  (2024-06-11T13:29:34Z)
• Forecasting long-time dynamics in quantum many-body systems by dynamic mode decomposition [6.381013699474244]
  We use short-time data of physical quantities to forecast long-time behavior of strongly entangled systems.<n>The method, though simple, enables accurate forecasts amazingly at time as long as nearly an order of magnitude longer than that of the short-time training data.
  arXiv  Detail & Related papers  (2024-03-29T03:10:34Z)
• Attractor Memory for Long-Term Time Series Forecasting: A Chaos Perspective [63.60312929416228]
  textbftextitAttraos incorporates chaos theory into long-term time series forecasting. We show that Attraos outperforms various LTSF methods on mainstream datasets and chaotic datasets with only one-twelfth of the parameters compared to PatchTST.
  arXiv  Detail & Related papers  (2024-02-18T05:35:01Z)
• Towards Long-Term Time-Series Forecasting: Feature, Pattern, and Distribution [57.71199089609161]
  Long-term time-series forecasting (LTTF) has become a pressing demand in many applications, such as wind power supply planning. Transformer models have been adopted to deliver high prediction capacity because of the high computational self-attention mechanism. We propose an efficient Transformerbased model, named Conformer, which differentiates itself from existing methods for LTTF in three aspects.
  arXiv  Detail & Related papers  (2023-01-05T13:59:29Z)
• Physics-Inspired Temporal Learning of Quadrotor Dynamics for Accurate Model Predictive Trajectory Tracking [76.27433308688592]
  Accurately modeling quadrotor's system dynamics is critical for guaranteeing agile, safe, and stable navigation. We present a novel Physics-Inspired Temporal Convolutional Network (PI-TCN) approach to learning quadrotor's system dynamics purely from robot experience. Our approach combines the expressive power of sparse temporal convolutions and dense feed-forward connections to make accurate system predictions.
  arXiv  Detail & Related papers  (2022-06-07T13:51:35Z)
• Multivariate Time Series Forecasting with Dynamic Graph Neural ODEs [65.18780403244178]
  We propose a continuous model to forecast Multivariate Time series with dynamic Graph neural Ordinary Differential Equations (MTGODE) Specifically, we first abstract multivariate time series into dynamic graphs with time-evolving node features and unknown graph structures. Then, we design and solve a neural ODE to complement missing graph topologies and unify both spatial and temporal message passing.
  arXiv  Detail & Related papers  (2022-02-17T02:17:31Z)
• Stochastically forced ensemble dynamic mode decomposition for forecasting and analysis of near-periodic systems [65.44033635330604]
  We introduce a novel load forecasting method in which observed dynamics are modeled as a forced linear system. We show that its use of intrinsic linear dynamics offers a number of desirable properties in terms of interpretability and parsimony. Results are presented for a test case using load data from an electrical grid.
  arXiv  Detail & Related papers  (2020-10-08T20:25:52Z)
• Deep Switching Auto-Regressive Factorization:Application to Time Series Forecasting [16.934920617960085]
  DSARF approximates high dimensional data by a product variables between time dependent weights and spatially dependent factors. DSARF is different from the state-of-the-art techniques in that it parameterizes the weights in terms of a deep switching vector auto-regressive factorization. Our experiments attest the superior performance of DSARF in terms of long- and short-term prediction error, when compared with the state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-09-10T20:15:59Z)
• Supporting Optimal Phase Space Reconstructions Using Neural Network Architecture for Time Series Modeling [68.8204255655161]
  We propose an artificial neural network with a mechanism to implicitly learn the phase spaces properties. Our approach is either as competitive as or better than most state-of-the-art strategies.
  arXiv  Detail & Related papers  (2020-06-19T21:04:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.