Towards Scaling Law Analysis For Spatiotemporal Weather Data

• URL: http://arxiv.org/abs/2604.05068v1
• Date: Mon, 06 Apr 2026 18:14:53 GMT
• Title: Towards Scaling Law Analysis For Spatiotemporal Weather Data
• Abstract summary: We extend neural scaling analysis for autoregressive weather forecasting from single-step training loss to long rollouts and per-channel metrics.<n>We find strong cross-channel and cross-horizon heterogeneity: pooled scaling can look favorable while many channels degrade at late leads.<n>We discuss implications for weighted objectives, horizon-aware curricula, and resource allocation across outputs.
• Score: 50.25336689856158
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Compute-optimal scaling laws are relatively well studied for NLP and CV, where objectives are typically single-step and targets are comparatively homogeneous. Weather forecasting is harder to characterize in the same framework: autoregressive rollouts compound errors over long horizons, outputs couple many physical channels with disparate scales and predictability, and globally pooled test metrics can disagree sharply with per-channel, late-lead behavior implied by short-horizon training. We extend neural scaling analysis for autoregressive weather forecasting from single-step training loss to long rollouts and per-channel metrics. We quantify (1) how prediction error is distributed across channels and how its growth rate evolves with forecast horizon, (2) if power law scaling holds for test error, relative to rollout length when error is pooled globally, and (3) how that fit varies jointly with horizon and channel for parameter, data, and compute-based scaling axes. We find strong cross-channel and cross-horizon heterogeneity: pooled scaling can look favorable while many channels degrade at late leads. We discuss implications for weighted objectives, horizon-aware curricula, and resource allocation across outputs.

Related papers

• Long-Time Trajectory Approximation via SA-NODEs: Model Predictive and Floquet Strategies [0.49655032077676126]
  We study the approximation of dynamical systems by semi-autonomous neural ordinary differential equations (SA-NODEs) over long time horizons.<n>For a single network trained on the whole horizon, the available error bound deteriorates double exponentially in the horizon length.<n>We develop two training strategies that avoid this barrier, each built on a reset of the state.
  arXiv  Detail & Related papers  (2026-08-11T09:55:54Z)
• Nipping the Butterfly Effect in the Bud: Self-Output Fine-Tuning for Autoregressive Weather Prediction [7.601516977968089]
  We study the error growth phenomenon from both theoretical and empirical perspectives.<n>Our analysis reveals that the growth is driven by a feedback loop between output errors and input distribution shifts.<n>To mitigate this issue, we propose textbfSelf-Output Fine-Tuning (SOFT), a plug-and-play strategy that leverages the model's own one-step predictions.
  arXiv  Detail & Related papers  (2026-07-23T09:10:11Z)
• When Do Autoregressive Sequence Models Forecast Physical Wavefields? A Controlled Study on Synthetic Seismograms [43.8784307709823]
  Long-horizon autoregressive forecasting of physical signals is limited by error accumulation.<n>We ask when such rollout stays stable using synthetic three-component seismograms as a physically structured testbed.
  arXiv  Detail & Related papers  (2026-06-09T13:46:14Z)
• FEG-Pro: Forecast-Error Growth Profiling for Finite-Horizon Instability Analysis of Nonlinear Time Series [0.0]
  We propose FEG-Pro, a forecast-error growth profiling framework for nonlinear scalar time series.<n>The method constructs autocorrelation-guided sparse histories, performs distance-weighted k-nearest-neighbor multi-horizon forecasting, and analyzes the logarithmic growth of geometrically averaged forecast errors.<n>The pipeline also extracts the formal fit-selection regime, curvature, residual roughness after quadratic detrending, monotonicity, and forecast-error distribution entropy (FEDE) from signed multi-horizon errors.
  arXiv  Detail & Related papers  (2026-05-17T06:38:37Z)
• Adaptive Spectral Feature Forecasting for Diffusion Sampling Acceleration [58.19554276924402]
  We propose spectral diffusion feature forecaster (Spectrum) to enable global, long-range feature reuse with tightly controlled error.<n>We achieve up to 4.79$times$ speedup on FLUX.1 and 4.67$times$ speedup on Wan2.1-14B, while maintaining much higher sample quality compared with the baselines.
  arXiv  Detail & Related papers  (2026-03-02T08:59:11Z)
• Deep Time-series Forecasting Needs Kernelized Moment Balancing [56.619037429652984]
  Deep time-series forecasting can be formulated as a distribution balancing problem aimed at aligning the distribution of the forecasts and ground truths.<n>We propose direct forecasting with kernelized moment balancing (KMB-DF)<n>Experiments across multiple models and datasets show that KMB-DF consistently improves forecasting accuracy and achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-01-31T13:20:18Z)
• Tackling Incomplete Data in Air Quality Prediction: A Bayesian Deep Learning Framework for Uncertainty Quantification [6.665519965608339]
  This research lays a foundation for reliable deep learning based temporal-temporal forecasting with incomplete observations in emerging sensing paradigms, such as real world vehicle borne mobile monitoring.
  arXiv  Detail & Related papers  (2025-11-04T01:42:00Z)
• S$^2$Transformer: Scalable Structured Transformers for Global Station Weather Forecasting [67.93713728260646]
  Existing time series forecasting methods often ignore or unidirectionally model spatial correlation when conducting large-scale global station forecasting.<n>This contradicts the nature underlying observations of the global weather system limiting forecast performance.<n>We propose a novel Structured Spatial Attention in this paper.<n>It partitions the spatial graph into a set of subgraphs and instantiates Intra-subgraph Attention to learn local spatial correlation within each subgraph.<n>It aggregates nodes into subgraph representations for message passing among the subgraphs via Inter-subgraph Attention -- considering both spatial proximity and global correlation.
  arXiv  Detail & Related papers  (2025-09-10T05:33:28Z)
• Data-Driven Forecasting of High-Dimensional Transient and Stationary Processes via Space-Time Projection [0.0]
  Space-Time Projection (STP) is introduced as a data-driven forecasting approach for high-dimensional and time-resolved data.<n>The method computes extended space-time proper modes from training data spanning a prediction horizon comprising both hindcast and forecast intervals.
  arXiv  Detail & Related papers  (2025-03-31T03:36:59Z)
• Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape Perspective [66.80315289020487]
  Warmup-Stable-Decay (WSD) schedule uses a constant learning rate to produce a main branch of iterates that can continue indefinitely without a pre-specified compute budget.<n>We show that pretraining loss exhibits a river valley landscape, which resembles a deep valley with a river at its bottom.<n>Inspired by the theory, we introduce WSD-S, a variant of WSD that reuses previous checkpoints' decay phases and keeps only one main branch.
  arXiv  Detail & Related papers  (2024-10-07T16:49:39Z)
• From Reactive to Proactive Volatility Modeling with Hemisphere Neural Networks [0.0]
  We reinvigorate maximum likelihood estimation (MLE) for macroeconomic density forecasting through a novel neural network architecture with dedicated mean and variance hemispheres. Our Hemisphere Neural Network (HNN) provides proactive volatility forecasts based on leading indicators when it can, and reactive volatility based on the magnitude of previous prediction errors when it must.
  arXiv  Detail & Related papers  (2023-11-27T21:37:50Z)
• Building Autocorrelation-Aware Representations for Fine-Scale Spatiotemporal Prediction [1.2862507359003323]
  We present a novel deep learning architecture that incorporates theories of spatial statistics into neural networks. DeepLATTE contains an autocorrelation-guided semi-supervised learning strategy to enforce both local autocorrelation patterns and global autocorrelation trends. We conduct a demonstration of DeepLATTE using publicly available data for an important public health topic, air quality prediction in a well-fitting, complex physical environment.
  arXiv  Detail & Related papers  (2021-12-10T03:21:19Z)
• Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updates [67.19481956584465]
  It has been experimentally observed that the efficiency of distributed training with saturation (SGD) depends decisively on the batch size and -- in implementations -- on the staleness. We show that our results are tight and illustrate key findings in numerical experiments.
  arXiv  Detail & Related papers  (2021-03-03T12:08:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.