Learning Spatio-Temporal Foundation Models from Pure Synthetic Data

• URL: http://arxiv.org/abs/2607.16251v1
• Date: Sat, 27 Jun 2026 02:56:46 GMT
• Title: Learning Spatio-Temporal Foundation Models from Pure Synthetic Data
• Abstract summary: NeoST is a scalable synthetic-training corpus to real-world bias, a latent-space reasoning architecture that generates and iteratively refines multiple future trajectories, and latent-space objectives that emphasize structural dynamics and enable inference-time correction under distribution shifts.<n>Extensive experiments across diverse real-world benchmarks show that NeoST consistently outperforms existing STFMs in diverse real-worldtemporal systems.
• Score: 85.23980441738811
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Spatio-Temporal Foundation Models (STFMs) aim to learn generalizable representations of complex dynamical systems across space and time. However, existing approaches suffer from distributional bias in real-world pre-training data, structural bottlenecks of autoregressive or diffusion-based paradigms, and objectives that overemphasize point-wise reconstruction in noisy observation space.We propose \textbf{NeoST}, the first spatio-temporal foundation model pre-trained solely on procedurally generated synthetic systems. NeoST introduces a scalable synthetic pre-training corpus to mitigate real-world bias, a latent-space reasoning architecture that generates and iteratively refines multiple future trajectories without sequential error accumulation, and latent-space objectives that emphasize structural dynamics and enable inference-time correction under distribution shifts.Extensive experiments across diverse real-world benchmarks show that NeoST consistently outperforms existing STFMs in diverse real-world spatio-temporal systems, achieves superior long-horizon stability and inference efficiency.

Related papers

• STKAN: Kolmogorov-Arnold Networks for Spatio-Temporal Forecasting [30.14175741484625]
  We propose STKAN, a Taylor-nomial forecasting architecture that introduces Kolmogo-Arnold Network modules into spatial and temporal token mixing.<n>Experiments show that STKAN achieves better than evaluated spatial interactions benchmarks on five settings tested.<n>These results suggest that the design of nonlinear function approximators can serve as a useful complement to architectural design in traffic-temporal forecasting.
  arXiv  Detail & Related papers  (2026-07-14T10:18:54Z)
• Decoupled Latent Optimization of Diffusion Models for Full Waveform Inversion [4.293965179144764]
  Full waveform inversion (FWI) recovers subsurface velocity from recordings by solving a severely ill-posed, nonconstrained quadratic PDE-velocity optimization.<n>Recent diffusion-prior methods improve realism at the cost of a fragile trade-off between the standard formulation and prior consistency.<n>We propose De Latent Optimization (DLO), which relaxes the latent-penalty objective into a latent-penalty objective over an auxiliary physical variable and a latent variable.<n>On the OpenFWI benchmark, DLO outperforms classical regularizers and existing diffusion-based methods under clean, noisy, and missing-trace.
  arXiv  Detail & Related papers  (2026-06-12T05:55:10Z)
• EIDOS: Latent-Space Predictive Learning for Time Series Foundation Models [37.917978019436674]
  EIDOS is a foundation model family that shifts pretraining from future value prediction to latent-space predictive learning.<n>We train a causal Transformer to predict the evolution of latent representations, encouraging the emergence of structured and temporally coherent latent states.
  arXiv  Detail & Related papers  (2026-02-15T07:07:20Z)
• Scalable Spatio-Temporal SE(3) Diffusion for Long-Horizon Protein Dynamics [51.85385061275941]
  Molecular dynamics (MD) simulations remain the gold standard for studying protein dynamics.<n>Recent generative models have shown promise in accelerating simulations, yet they struggle with long-horizon generation.<n>We present STAR-MD, a scalable diffusion model that generates physically plausible protein trajectories over micro-scale timescales.
  arXiv  Detail & Related papers  (2026-02-02T14:13:28Z)
• Online Bayesian Experimental Design for Partially Observed Dynamical Systems [10.774974720491565]
  We develop a principled framework for optimizing data collection in dynamical systems with partial observability.<n>Our framework successfully handles both partial observability and online inference.
  arXiv  Detail & Related papers  (2025-11-06T14:29:05Z)
• A Deep Learning Approach for Spatio-Temporal Forecasting of InSAR Ground Deformation in Eastern Ireland [2.840858735842673]
  Monitoring ground displacement is crucial for urban infrastructure and mitigating geological hazards.<n>This paper introduces a novel deep learning framework that transforms sparse point measurements into a dense-temporal tensor.<n>Results demonstrate that the proposed architecture provides more accurate and spatially coherent forecasts.
  arXiv  Detail & Related papers  (2025-09-17T17:10:18Z)
• Learning to Dissipate Energy in Oscillatory State-Space Models [51.98491034847041]
  State-space models (SSMs) are a class of networks for sequence learning.<n>We show that D-LinOSS consistently outperforms previous LinOSS methods on long-range learning tasks.
  arXiv  Detail & Related papers  (2025-05-17T23:15:17Z)
• Attractor Memory for Long-Term Time Series Forecasting: A Chaos Perspective [63.60312929416228]
  textbftextitAttraos incorporates chaos theory into long-term time series forecasting. We show that Attraos outperforms various LTSF methods on mainstream datasets and chaotic datasets with only one-twelfth of the parameters compared to PatchTST.
  arXiv  Detail & Related papers  (2024-02-18T05:35:01Z)
• OpenSTL: A Comprehensive Benchmark of Spatio-Temporal Predictive Learning [67.07363529640784]
  We propose OpenSTL to categorize prevalent approaches into recurrent-based and recurrent-free models. We conduct standard evaluations on datasets across various domains, including synthetic moving object trajectory, human motion, driving scenes, traffic flow and forecasting weather. We find that recurrent-free models achieve a good balance between efficiency and performance than recurrent models.
  arXiv  Detail & Related papers  (2023-06-20T03:02:14Z)
• Supporting Optimal Phase Space Reconstructions Using Neural Network Architecture for Time Series Modeling [68.8204255655161]
  We propose an artificial neural network with a mechanism to implicitly learn the phase spaces properties. Our approach is either as competitive as or better than most state-of-the-art strategies.
  arXiv  Detail & Related papers  (2020-06-19T21:04:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.