Stochastic Latent Transformer: Efficient Modelling of Stochastically Forced Zonal Jets

• URL: http://arxiv.org/abs/2310.16741v2
• Date: Mon, 18 Dec 2023 14:27:34 GMT
• Title: Stochastic Latent Transformer: Efficient Modelling of Stochastically Forced Zonal Jets
• Authors: Ira J. S. Shokar, Rich R. Kerswell, Peter H. Haynes
• Abstract summary: We present a novel deep probabilistic learning approach, the 'Stochastic Latent Transformer' (SLT) The SLT accurately reproduces system dynamics across various integration periods, validated through quantitative diagnostics. It achieves a five-order-of-magnitude speedup in emulating the zonally-averaged flow.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a novel probabilistic deep learning approach, the 'Stochastic Latent Transformer' (SLT), designed for the efficient reduced-order modelling of stochastic partial differential equations. Stochastically driven flow models are pertinent to a diverse range of natural phenomena, including jets on giant planets, ocean circulation, and the variability of midlatitude weather. However, much of the recent progress in deep learning has predominantly focused on deterministic systems. The SLT comprises a stochastically-forced transformer paired with a translation-equivariant autoencoder, trained towards the Continuous Ranked Probability Score. We showcase its effectiveness by applying it to a well-researched zonal jet system, where the interaction between stochastically forced eddies and the zonal mean flow results in a rich low-frequency variability. The SLT accurately reproduces system dynamics across various integration periods, validated through quantitative diagnostics that include spectral properties and the rate of transitions between distinct states. The SLT achieves a five-order-of-magnitude speedup in emulating the zonally-averaged flow compared to direct numerical simulations. This acceleration facilitates the cost-effective generation of large ensembles, enabling the exploration of statistical questions concerning the probabilities of spontaneous transition events.

Related papers

• Capturing Extreme Events in Turbulence using an Extreme Variational Autoencoder (xVAE) [2.393499494583]
  Extreme variational Autoencoder (xVAE) embeds a max-infinitely divisible process with heavy-tailed distributions into a standard VAE framework. xVAE reduces system dimensionality by learning non-linear latent representations of data.
  arXiv  Detail & Related papers  (2025-02-07T06:17:31Z)
• Implicit factorized transformer approach to fast prediction of turbulent channel flows [6.70175842351963]
  We introduce a modified implicit factorized transformer (IFactFormer-m) model which replaces the original chained factorized attention with parallel factorized attention. The IFactFormer-m model successfully performs long-term predictions for turbulent channel flow.
  arXiv  Detail & Related papers  (2024-12-25T09:05:14Z)
• FlowTS: Time Series Generation via Rectified Flow [67.41208519939626]
  FlowTS is an ODE-based model that leverages rectified flow with straight-line transport in probability space. For unconditional setting, FlowTS achieves state-of-the-art performance, with context FID scores of 0.019 and 0.011 on Stock and ETTh datasets. For conditional setting, we have achieved superior performance in solar forecasting.
  arXiv  Detail & Related papers  (2024-11-12T03:03:23Z)
• Learning Controlled Stochastic Differential Equations [61.82896036131116]
  This work proposes a novel method for estimating both drift and diffusion coefficients of continuous, multidimensional, nonlinear controlled differential equations with non-uniform diffusion. We provide strong theoretical guarantees, including finite-sample bounds for (L2), (Linfty), and risk metrics, with learning rates adaptive to coefficients' regularity. Our method is available as an open-source Python library.
  arXiv  Detail & Related papers  (2024-11-04T11:09:58Z)
• On the Trajectory Regularity of ODE-based Diffusion Sampling [79.17334230868693]
  Diffusion-based generative models use differential equations to establish a smooth connection between a complex data distribution and a tractable prior distribution. In this paper, we identify several intriguing trajectory properties in the ODE-based sampling process of diffusion models.
  arXiv  Detail & Related papers  (2024-05-18T15:59:41Z)
• Variational Schrödinger Diffusion Models [14.480273869571468]
  Schr"odinger bridge (SB) has emerged as the go-to method for optimizing transportation plans in diffusion models. We leverage variational inference to linearize the forward score functions (variational scores) of SB. We propose the variational Schr"odinger diffusion model (VSDM), where the forward process is a multivariate diffusion and the variational scores are adaptively optimized for efficient transport.
  arXiv  Detail & Related papers  (2024-05-08T04:01:40Z)
• Bayesian Conditional Diffusion Models for Versatile Spatiotemporal Turbulence Generation [13.278744447861289]
  We introduce a novel generative framework grounded in probabilistic diffusion models for turbulence generation. A notable feature of our approach is the proposed method for long-span flow sequence generation, which is based on autoregressive-based conditional sampling. We showcase the versatile turbulence generation capability of our framework through a suite of numerical experiments.
  arXiv  Detail & Related papers  (2023-11-14T04:08:14Z)
• Learning noise-induced transitions by multi-scaling reservoir computing [2.9170682727903863]
  We develop a machine learning model, reservoir computing as a type of recurrent neural network, to learn noise-induced transitions. The trained model generates accurate statistics of transition time and the number of transitions. It is also aware of the asymmetry of the double-well potential, the rotational dynamics caused by non-detailed balance, and transitions in multi-stable systems.
  arXiv  Detail & Related papers  (2023-09-11T12:26:36Z)
• A Geometric Perspective on Diffusion Models [57.27857591493788]
  We inspect the ODE-based sampling of a popular variance-exploding SDE. We establish a theoretical relationship between the optimal ODE-based sampling and the classic mean-shift (mode-seeking) algorithm.
  arXiv  Detail & Related papers  (2023-05-31T15:33:16Z)
• Manifold Interpolating Optimal-Transport Flows for Trajectory Inference [64.94020639760026]
  We present a method called Manifold Interpolating Optimal-Transport Flow (MIOFlow) MIOFlow learns, continuous population dynamics from static snapshot samples taken at sporadic timepoints. We evaluate our method on simulated data with bifurcations and merges, as well as scRNA-seq data from embryoid body differentiation, and acute myeloid leukemia treatment.
  arXiv  Detail & Related papers  (2022-06-29T22:19:03Z)
• Haar Wavelet based Block Autoregressive Flows for Trajectories [129.37479472754083]
  Prediction of trajectories such as that of pedestrians is crucial to the performance of autonomous agents. We introduce a novel Haar wavelet based block autoregressive model leveraging split couplings. We illustrate the advantages of our approach for generating diverse and accurate trajectories on two real-world datasets.
  arXiv  Detail & Related papers  (2020-09-21T13:57:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.