climt-paraformer: Stable Emulation of Convective Parameterization using a Temporal Memory-aware Transformer

• URL: http://arxiv.org/abs/2604.21085v1
• Date: Wed, 22 Apr 2026 20:55:35 GMT
• Title: climt-paraformer: Stable Emulation of Convective Parameterization using a Temporal Memory-aware Transformer
• Abstract summary: We develop a temporal memory-aware Transformer emulator for the Emanuel convective parameterization.<n>The Transformer captures temporal correlations and nonlinear interactions across consecutive atmospheric states.
• Score: 3.8491907908078917
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurate representation of moist convective sub-grid-scale processes remains a major challenge in global climate models, as traditional parameterization schemes are both computationally expensive and difficult to scale. Neural network (NN) emulators offer a promising alternative by learning efficient mappings between atmospheric states and convective tendencies while retaining fidelity to the underlying physics. However, most existing NN-based parameterizations are memory-less and rely only on instantaneous inputs, even though convection evolves over time and depends on prior atmospheric states. Recent studies have begun to incorporate convective memory, but they often treat past states as independent features rather than modeling temporal dependencies explicitly. In this work, we develop a temporal memory-aware Transformer emulator for the Emanuel convective parameterization and evaluate it in a single-column climate model (SCM) under both offline and online configurations. The Transformer captures temporal correlations and nonlinear interactions across consecutive atmospheric states. Compared with baseline emulators, including a memory-less multilayer perceptron and a recurrent long short-term memory model, the Transformer achieves lower offline errors. Sensitivity analysis indicates that a memory length of approximately 100 minutes yields the best performance, whereas longer memory degrades performance. We further test the emulator in long-term coupled simulations and show that it remains stable over 10 years. Overall, this study demonstrates the importance of explicit temporal modeling for NN-based parameterizations.

Related papers

• MEMTS: Internalizing Domain Knowledge via Parameterized Memory for Retrieval-Free Domain Adaptation of Time Series Foundation Models [51.506429027626005]
  Memory for Time Series (MEMTS) is a lightweight and plug-and-play method for retrieval-free domain adaptation in time series forecasting.<n>Key component of MEMTS is a Knowledge Persistence Module (KPM), which internalizes domain-specific temporal dynamics.<n>This paradigm shift enables MEMTS to achieve accurate domain adaptation with constant-time inference and near-zero latency.
  arXiv  Detail & Related papers  (2026-02-14T14:00:06Z)
• Video Prediction of Dynamic Physical Simulations With Pixel-Space Spatiotemporal Transformers [3.951575888190684]
  This study investigates a transformer adaptation for video prediction with a simple end-to-end approach, comparing various selftemporal-attention layouts.<n>We introduce a simple yet effective transformer for autoregressive video prediction, utilizing continuous pixel-space representations for video prediction horizon.
  arXiv  Detail & Related papers  (2025-10-23T17:58:45Z)
• EntroPE: Entropy-Guided Dynamic Patch Encoder for Time Series Forecasting [50.794700596484894]
  We propose EntroPE (Entropy-Guided Dynamic Patch), a novel, temporally informed framework that dynamically detects transition points via conditional entropy.<n>This preserves temporal structure while retaining the computational benefits of patching.<n> Experiments across long-term forecasting benchmarks demonstrate that EntroPE improves both accuracy and efficiency.
  arXiv  Detail & Related papers  (2025-09-30T12:09:56Z)
• Improving Long-term Autoregressive Spatiotemporal Predictions: A Proof of Concept with Fluid Dynamics [10.71350538032054]
  For complex systems, long-term accuracy often deteriorates due to error accumulation.<n>We propose the PushForward framework, which retains one-step-ahead training while enabling multi-step learning.<n> SPF builds a supplementary dataset from model predictions and combines it with ground truth via an acquisition strategy.
  arXiv  Detail & Related papers  (2025-08-25T23:51:18Z)
• MesaNet: Sequence Modeling by Locally Optimal Test-Time Training [67.45211108321203]
  We introduce a numerically stable, chunkwise parallelizable version of the recently proposed Mesa layer.<n>We show that optimal test-time training enables reaching lower language modeling perplexity and higher downstream benchmark performance than previous RNNs.
  arXiv  Detail & Related papers  (2025-06-05T16:50:23Z)
• Hybrid Neural-MPM for Interactive Fluid Simulations in Real-Time [57.30651532625017]
  We present a novel hybrid method that integrates numerical simulation, neural physics, and generative control.<n>Our system demonstrates robust performance across diverse 2D/3D scenarios, material types, and obstacle interactions.<n>We promise to release both models and data upon acceptance.
  arXiv  Detail & Related papers  (2025-05-25T01:27:18Z)
• Reduced Order Modeling with Shallow Recurrent Decoder Networks [5.686433280542813]
  SHRED-ROM is a robust decoding-only strategy that encodes the numerically unstable approximation of an inverse.<n>We show that SHRED-ROM accurately reconstructs the state dynamics for new parameter values starting from limited fixed or mobile sensors.
  arXiv  Detail & Related papers  (2025-02-15T23:41:31Z)
• Parsimony or Capability? Decomposition Delivers Both in Long-term Time Series Forecasting [46.63798583414426]
  Long-term time series forecasting (LTSF) represents a critical frontier in time series analysis. Our study demonstrates, through both analytical and empirical evidence, that decomposition is key to containing excessive model inflation. Remarkably, by tailoring decomposition to the intrinsic dynamics of time series data, our proposed model outperforms existing benchmarks.
  arXiv  Detail & Related papers  (2024-01-22T13:15:40Z)
• Convolutional State Space Models for Long-Range Spatiotemporal Modeling [65.0993000439043]
  ConvS5 is an efficient variant for long-rangetemporal modeling. It significantly outperforms Transformers and ConvNISTTM on a long horizon Moving-Lab experiment while training 3X faster than ConvLSTM and generating samples 400X faster than Transformers.
  arXiv  Detail & Related papers  (2023-10-30T16:11:06Z)
• Navigating the Noise: Bringing Clarity to ML Parameterization Design with O(100) Ensembles [1.9199275795132604]
  Machine-learning (ML) parameterizations of subgrid processes may one day replace conventional parameterizations.<n>Uncertainty about the relationship between offline and online performance hinders their development.<n>We show how systematic reductions in offline error manifest in changes to online error and online stability.
  arXiv  Detail & Related papers  (2023-09-28T05:34:29Z)
• Memory-and-Anticipation Transformer for Online Action Understanding [52.24561192781971]
  We propose a novel memory-anticipation-based paradigm to model an entire temporal structure, including the past, present, and future. We present Memory-and-Anticipation Transformer (MAT), a memory-anticipation-based approach, to address the online action detection and anticipation tasks.
  arXiv  Detail & Related papers  (2023-08-15T17:34:54Z)
• Temporal Subsampling Diminishes Small Spatial Scales in Recurrent Neural Network Emulators of Geophysical Turbulence [0.0]
  We investigate how an often overlooked processing step affects the quality of an emulator's predictions. We implement ML architectures from a class of methods called reservoir computing: (1) a form of spatial Vector Autoregression (N VAR), and (2) an Echo State Network (ESN) In all cases, subsampling the training data consistently leads to an increased bias at small scales that resembles numerical diffusion.
  arXiv  Detail & Related papers  (2023-04-28T21:34:53Z)
• Alternating ConvLSTM: Learning Force Propagation with Alternate State Updates [29.011464047344614]
  We introduce the alternating convolutional Long Short-Term Memory (Alt-ConvLSTM) that models the force propagation mechanisms in a deformable object with near-uniform material properties. We demonstrate how this novel scheme imitates the alternate updates of the first and second-order terms in the forward method of numerical PDE solvers. We validate our Alt-ConvLSTM on human soft tissue simulation with thousands of particles and consistent body pose changes.
  arXiv  Detail & Related papers  (2020-06-14T06:43:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.