DYffusion: A Dynamics-informed Diffusion Model for Spatiotemporal Forecasting

• URL: http://arxiv.org/abs/2306.01984v2
• Date: Wed, 11 Oct 2023 07:35:27 GMT
• Title: DYffusion: A Dynamics-informed Diffusion Model for Spatiotemporal Forecasting
• Authors: Salva R\"uhling Cachay, Bo Zhao, Hailey Joren, Rose Yu
• Abstract summary: We propose an approach for efficiently training diffusion models for probabilistic forecasting. We train a time-conditioned interpolator and a forecaster network that mimic the forward and reverse processes of standard diffusion models. Our approach performs competitively on probabilistic forecasting of complex dynamics in sea surface temperatures, Navier-Stokes flows, and flows spring systems.
• Score: 18.86526240105348
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While diffusion models can successfully generate data and make predictions, they are predominantly designed for static images. We propose an approach for efficiently training diffusion models for probabilistic spatiotemporal forecasting, where generating stable and accurate rollout forecasts remains challenging, Our method, DYffusion, leverages the temporal dynamics in the data, directly coupling it with the diffusion steps in the model. We train a stochastic, time-conditioned interpolator and a forecaster network that mimic the forward and reverse processes of standard diffusion models, respectively. DYffusion naturally facilitates multi-step and long-range forecasting, allowing for highly flexible, continuous-time sampling trajectories and the ability to trade-off performance with accelerated sampling at inference time. In addition, the dynamics-informed diffusion process in DYffusion imposes a strong inductive bias and significantly improves computational efficiency compared to traditional Gaussian noise-based diffusion models. Our approach performs competitively on probabilistic forecasting of complex dynamics in sea surface temperatures, Navier-Stokes flows, and spring mesh systems.

Related papers

• Energy-Based Diffusion Language Models for Text Generation [126.23425882687195]
  Energy-based Diffusion Language Model (EDLM) is an energy-based model operating at the full sequence level for each diffusion step. Our framework offers a 1.3$times$ sampling speedup over existing diffusion models.
  arXiv  Detail & Related papers  (2024-10-28T17:25:56Z)
• MG-TSD: Multi-Granularity Time Series Diffusion Models with Guided Learning Process [26.661721555671626]
  We introduce a novel Multi-Granularity Time Series (MG-TSD) model, which achieves state-of-the-art predictive performance. Our approach does not rely on additional external data, making it versatile and applicable across various domains.
  arXiv  Detail & Related papers  (2024-03-09T01:15:03Z)
• Weather Prediction with Diffusion Guided by Realistic Forecast Processes [49.07556359513563]
  We introduce a novel method that applies diffusion models (DM) for weather forecasting. Our method can achieve both direct and iterative forecasting with the same modeling framework. The flexibility and controllability of our model empowers a more trustworthy DL system for the general weather community.
  arXiv  Detail & Related papers  (2024-02-06T21:28:42Z)
• Data Attribution for Diffusion Models: Timestep-induced Bias in Influence Estimation [53.27596811146316]
  Diffusion models operate over a sequence of timesteps instead of instantaneous input-output relationships in previous contexts. We present Diffusion-TracIn that incorporates this temporal dynamics and observe that samples' loss gradient norms are highly dependent on timestep. We introduce Diffusion-ReTrac as a re-normalized adaptation that enables the retrieval of training samples more targeted to the test sample of interest.
  arXiv  Detail & Related papers  (2024-01-17T07:58:18Z)
• Non-Denoising Forward-Time Diffusions [4.831663144935879]
  We show that the time-reversal argument, common to all denoising diffusion probabilistic modeling proposals, is not necessary. We obtain diffusion processes targeting the desired data distribution by taking appropriate mixtures of diffusion bridges. We develop a unifying view of the drift adjustments corresponding to our and to time-reversal approaches.
  arXiv  Detail & Related papers  (2023-12-22T10:26:31Z)
• Fast Sampling via Discrete Non-Markov Diffusion Models [49.598085130313514]
  We propose a discrete non-Markov diffusion model, which admits an accelerated reverse sampling for discrete data generation. Our method significantly reduces the number of function evaluations (i.e., calls to the neural network), making the sampling process much faster.
  arXiv  Detail & Related papers  (2023-12-14T18:14:11Z)
• Generative Modeling with Phase Stochastic Bridges [49.4474628881673]
  Diffusion models (DMs) represent state-of-the-art generative models for continuous inputs. We introduce a novel generative modeling framework grounded in textbfphase space dynamics Our framework demonstrates the capability to generate realistic data points at an early stage of dynamics propagation.
  arXiv  Detail & Related papers  (2023-10-11T18:38:28Z)
• Benchmarking Autoregressive Conditional Diffusion Models for Turbulent Flow Simulation [29.806100463356906]
  We analyze if fully data-driven fluid solvers that utilize an autoregressive rollout based on conditional diffusion models are a viable option. We investigate accuracy, posterior sampling, spectral behavior, and temporal stability, while requiring that methods generalize to flow parameters beyond the training regime. We find that even simple diffusion-based approaches can outperform multiple established flow prediction methods in terms of accuracy and temporal stability, while being on par with state-of-the-art stabilization techniques like unrolling at training time.
  arXiv  Detail & Related papers  (2023-09-04T18:01:42Z)
• A prior regularized full waveform inversion using generative diffusion models [0.5156484100374059]
  Full waveform inversion (FWI) has the potential to provide high-resolution subsurface model estimations. Due to limitations in observation, e.g., regional noise, limited shots or receivers, and band-limited data, it is hard to obtain the desired high-resolution model with FWI. We propose a new paradigm for FWI regularized by generative diffusion models.
  arXiv  Detail & Related papers  (2023-06-22T10:10:34Z)
• How Much is Enough? A Study on Diffusion Times in Score-based Generative Models [76.76860707897413]
  Current best practice advocates for a large T to ensure that the forward dynamics brings the diffusion sufficiently close to a known and simple noise distribution. We show how an auxiliary model can be used to bridge the gap between the ideal and the simulated forward dynamics, followed by a standard reverse diffusion process.
  arXiv  Detail & Related papers  (2022-06-10T15:09:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.