Stochastic Latent Transformer: Efficient Modelling of Stochastically
Forced Zonal Jets
- URL: http://arxiv.org/abs/2310.16741v2
- Date: Mon, 18 Dec 2023 14:27:34 GMT
- Title: Stochastic Latent Transformer: Efficient Modelling of Stochastically
Forced Zonal Jets
- Authors: Ira J. S. Shokar, Rich R. Kerswell, Peter H. Haynes
- Abstract summary: We present a novel deep probabilistic learning approach, the 'Stochastic Latent Transformer' (SLT)
The SLT accurately reproduces system dynamics across various integration periods, validated through quantitative diagnostics.
It achieves a five-order-of-magnitude speedup in emulating the zonally-averaged flow.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We present a novel probabilistic deep learning approach, the 'Stochastic
Latent Transformer' (SLT), designed for the efficient reduced-order modelling
of stochastic partial differential equations. Stochastically driven flow models
are pertinent to a diverse range of natural phenomena, including jets on giant
planets, ocean circulation, and the variability of midlatitude weather.
However, much of the recent progress in deep learning has predominantly focused
on deterministic systems. The SLT comprises a stochastically-forced transformer
paired with a translation-equivariant autoencoder, trained towards the
Continuous Ranked Probability Score. We showcase its effectiveness by applying
it to a well-researched zonal jet system, where the interaction between
stochastically forced eddies and the zonal mean flow results in a rich
low-frequency variability. The SLT accurately reproduces system dynamics across
various integration periods, validated through quantitative diagnostics that
include spectral properties and the rate of transitions between distinct
states. The SLT achieves a five-order-of-magnitude speedup in emulating the
zonally-averaged flow compared to direct numerical simulations. This
acceleration facilitates the cost-effective generation of large ensembles,
enabling the exploration of statistical questions concerning the probabilities
of spontaneous transition events.
Related papers
- On the Trajectory Regularity of ODE-based Diffusion Sampling [79.17334230868693]
Diffusion-based generative models use differential equations to establish a smooth connection between a complex data distribution and a tractable prior distribution.
In this paper, we identify several intriguing trajectory properties in the ODE-based sampling process of diffusion models.
arXiv Detail & Related papers (2024-05-18T15:59:41Z) - Variational Schrödinger Diffusion Models [14.480273869571468]
Schr"odinger bridge (SB) has emerged as the go-to method for optimizing transportation plans in diffusion models.
We leverage variational inference to linearize the forward score functions (variational scores) of SB.
We propose the variational Schr"odinger diffusion model (VSDM), where the forward process is a multivariate diffusion and the variational scores are adaptively optimized for efficient transport.
arXiv Detail & Related papers (2024-05-08T04:01:40Z) - Equivariant Flow Matching with Hybrid Probability Transport [69.11915545210393]
Diffusion Models (DMs) have demonstrated effectiveness in generating feature-rich geometries.
DMs typically suffer from unstable probability dynamics with inefficient sampling speed.
We introduce geometric flow matching, which enjoys the advantages of both equivariant modeling and stabilized probability dynamics.
arXiv Detail & Related papers (2023-12-12T11:13:13Z) - Bayesian Conditional Diffusion Models for Versatile Spatiotemporal
Turbulence Generation [13.278744447861289]
We introduce a novel generative framework grounded in probabilistic diffusion models for turbulence generation.
A notable feature of our approach is the proposed method for long-span flow sequence generation, which is based on autoregressive-based conditional sampling.
We showcase the versatile turbulence generation capability of our framework through a suite of numerical experiments.
arXiv Detail & Related papers (2023-11-14T04:08:14Z) - Learning noise-induced transitions by multi-scaling reservoir computing [2.9170682727903863]
We develop a machine learning model, reservoir computing as a type of recurrent neural network, to learn noise-induced transitions.
The trained model generates accurate statistics of transition time and the number of transitions.
It is also aware of the asymmetry of the double-well potential, the rotational dynamics caused by non-detailed balance, and transitions in multi-stable systems.
arXiv Detail & Related papers (2023-09-11T12:26:36Z) - Benchmarking Autoregressive Conditional Diffusion Models for Turbulent
Flow Simulation [29.806100463356906]
We analyze if fully data-driven fluid solvers that utilize an autoregressive rollout based on conditional diffusion models are a viable option.
We investigate accuracy, posterior sampling, spectral behavior, and temporal stability, while requiring that methods generalize to flow parameters beyond the training regime.
We find that even simple diffusion-based approaches can outperform multiple established flow prediction methods in terms of accuracy and temporal stability, while being on par with state-of-the-art stabilization techniques like unrolling at training time.
arXiv Detail & Related papers (2023-09-04T18:01:42Z) - Machine learning in and out of equilibrium [58.88325379746631]
Our study uses a Fokker-Planck approach, adapted from statistical physics, to explore these parallels.
We focus in particular on the stationary state of the system in the long-time limit, which in conventional SGD is out of equilibrium.
We propose a new variation of Langevin dynamics (SGLD) that harnesses without replacement minibatching.
arXiv Detail & Related papers (2023-06-06T09:12:49Z) - A Geometric Perspective on Diffusion Models [60.69328526215776]
We inspect the ODE-based sampling of a popular variance-exploding SDE and reveal several intriguing structures of its sampling dynamics.
We establish a theoretical relationship between the optimal ODE-based sampling and the classic mean-shift (mode-seeking) algorithm.
arXiv Detail & Related papers (2023-05-31T15:33:16Z) - Manifold Interpolating Optimal-Transport Flows for Trajectory Inference [64.94020639760026]
We present a method called Manifold Interpolating Optimal-Transport Flow (MIOFlow)
MIOFlow learns, continuous population dynamics from static snapshot samples taken at sporadic timepoints.
We evaluate our method on simulated data with bifurcations and merges, as well as scRNA-seq data from embryoid body differentiation, and acute myeloid leukemia treatment.
arXiv Detail & Related papers (2022-06-29T22:19:03Z) - Low-Rank Hankel Tensor Completion for Traffic Speed Estimation [7.346671461427793]
We propose a purely data-driven and model-free solution to the traffic state estimation problem.
By imposing a low-rank assumption on this tensor structure, we can approximate characterize both global patterns and the unknown complex local dynamics.
We conduct numerical experiments on both synthetic simulation data and real-world high-resolution data, and our results demonstrate the effectiveness and superiority of the proposed model.
arXiv Detail & Related papers (2021-05-21T00:08:06Z) - Haar Wavelet based Block Autoregressive Flows for Trajectories [129.37479472754083]
Prediction of trajectories such as that of pedestrians is crucial to the performance of autonomous agents.
We introduce a novel Haar wavelet based block autoregressive model leveraging split couplings.
We illustrate the advantages of our approach for generating diverse and accurate trajectories on two real-world datasets.
arXiv Detail & Related papers (2020-09-21T13:57:10Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.