PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers
- URL: http://arxiv.org/abs/2308.05732v2
- Date: Sat, 21 Oct 2023 15:41:47 GMT
- Title: PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers
- Authors: Phillip Lippe, Bastiaan S. Veeling, Paris Perdikaris, Richard E.
Turner, Johannes Brandstetter
- Abstract summary: Time-dependent partial differential equations (PDEs) are ubiquitous in science and engineering.
Deep neural network based surrogates have gained increased interest.
- Score: 40.097474800631
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Time-dependent partial differential equations (PDEs) are ubiquitous in
science and engineering. Recently, mostly due to the high computational cost of
traditional solution techniques, deep neural network based surrogates have
gained increased interest. The practical utility of such neural PDE solvers
relies on their ability to provide accurate, stable predictions over long time
horizons, which is a notoriously hard problem. In this work, we present a
large-scale analysis of common temporal rollout strategies, identifying the
neglect of non-dominant spatial frequency information, often associated with
high frequencies in PDE solutions, as the primary pitfall limiting stable,
accurate rollout performance. Based on these insights, we draw inspiration from
recent advances in diffusion models to introduce PDE-Refiner; a novel model
class that enables more accurate modeling of all frequency components via a
multistep refinement process. We validate PDE-Refiner on challenging benchmarks
of complex fluid dynamics, demonstrating stable and accurate rollouts that
consistently outperform state-of-the-art models, including neural, numerical,
and hybrid neural-numerical architectures. We further demonstrate that
PDE-Refiner greatly enhances data efficiency, since the denoising objective
implicitly induces a novel form of spectral data augmentation. Finally,
PDE-Refiner's connection to diffusion models enables an accurate and efficient
assessment of the model's predictive uncertainty, allowing us to estimate when
the surrogate becomes inaccurate.
Related papers
- Physics-informed deep learning and compressive collocation for high-dimensional diffusion-reaction equations: practical existence theory and numerics [5.380276949049726]
We develop and analyze an efficient high-dimensional Partial Differential Equations solver based on Deep Learning (DL)
We show, both theoretically and numerically, that it can compete with a novel stable and accurate compressive spectral collocation method.
arXiv Detail & Related papers (2024-06-03T17:16:11Z) - Uncertainty Quantification for Forward and Inverse Problems of PDEs via
Latent Global Evolution [110.99891169486366]
We propose a method that integrates efficient and precise uncertainty quantification into a deep learning-based surrogate model.
Our method endows deep learning-based surrogate models with robust and efficient uncertainty quantification capabilities for both forward and inverse problems.
Our method excels at propagating uncertainty over extended auto-regressive rollouts, making it suitable for scenarios involving long-term predictions.
arXiv Detail & Related papers (2024-02-13T11:22:59Z) - Deep Equilibrium Based Neural Operators for Steady-State PDEs [100.88355782126098]
We study the benefits of weight-tied neural network architectures for steady-state PDEs.
We propose FNO-DEQ, a deep equilibrium variant of the FNO architecture that directly solves for the solution of a steady-state PDE.
arXiv Detail & Related papers (2023-11-30T22:34:57Z) - Multi-fidelity reduced-order surrogate modeling [5.346062841242067]
We present a new data-driven strategy that combines dimensionality reduction with multi-fidelity neural network surrogates.
We show that the onset of instabilities and transients are well captured by this surrogate technique.
arXiv Detail & Related papers (2023-09-01T08:16:53Z) - A Neural PDE Solver with Temporal Stencil Modeling [44.97241931708181]
Recent Machine Learning (ML) models have shown new promises in capturing important dynamics in high-resolution signals.
This study shows that significant information is often lost in the low-resolution down-sampled features.
We propose a new approach, which combines the strengths of advanced time-series sequence modeling and state-of-the-art neural PDE solvers.
arXiv Detail & Related papers (2023-02-16T06:13:01Z) - Learning PDE Solution Operator for Continuous Modeling of Time-Series [1.39661494747879]
This work presents a partial differential equation (PDE) based framework which improves the dynamics modeling capability.
We propose a neural operator that can handle time continuously without requiring iterative operations or specific grids of temporal discretization.
Our framework opens up a new way for a continuous representation of neural networks that can be readily adopted for real-world applications.
arXiv Detail & Related papers (2023-02-02T03:47:52Z) - EgPDE-Net: Building Continuous Neural Networks for Time Series
Prediction with Exogenous Variables [22.145726318053526]
Inter-series correlation and time dependence among variables are rarely considered in the present continuous methods.
We propose a continuous-time model for arbitrary-step prediction to learn an unknown PDE system.
arXiv Detail & Related papers (2022-08-03T08:34:31Z) - Learning to Accelerate Partial Differential Equations via Latent Global
Evolution [64.72624347511498]
Latent Evolution of PDEs (LE-PDE) is a simple, fast and scalable method to accelerate the simulation and inverse optimization of PDEs.
We introduce new learning objectives to effectively learn such latent dynamics to ensure long-term stability.
We demonstrate up to 128x reduction in the dimensions to update, and up to 15x improvement in speed, while achieving competitive accuracy.
arXiv Detail & Related papers (2022-06-15T17:31:24Z) - Closed-form Continuous-Depth Models [99.40335716948101]
Continuous-depth neural models rely on advanced numerical differential equation solvers.
We present a new family of models, termed Closed-form Continuous-depth (CfC) networks, that are simple to describe and at least one order of magnitude faster.
arXiv Detail & Related papers (2021-06-25T22:08:51Z) - Neural ODE Processes [64.10282200111983]
We introduce Neural ODE Processes (NDPs), a new class of processes determined by a distribution over Neural ODEs.
We show that our model can successfully capture the dynamics of low-dimensional systems from just a few data-points.
arXiv Detail & Related papers (2021-03-23T09:32:06Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.