A Neural PDE Solver with Temporal Stencil Modeling
- URL: http://arxiv.org/abs/2302.08105v1
- Date: Thu, 16 Feb 2023 06:13:01 GMT
- Title: A Neural PDE Solver with Temporal Stencil Modeling
- Authors: Zhiqing Sun, Yiming Yang, Shinjae Yoo
- Abstract summary: Recent Machine Learning (ML) models have shown new promises in capturing important dynamics in high-resolution signals.
This study shows that significant information is often lost in the low-resolution down-sampled features.
We propose a new approach, which combines the strengths of advanced time-series sequence modeling and state-of-the-art neural PDE solvers.
- Score: 44.97241931708181
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Numerical simulation of non-linear partial differential equations plays a
crucial role in modeling physical science and engineering phenomena, such as
weather, climate, and aerodynamics. Recent Machine Learning (ML) models trained
on low-resolution spatio-temporal signals have shown new promises in capturing
important dynamics in high-resolution signals, under the condition that the
models can effectively recover the missing details. However, this study shows
that significant information is often lost in the low-resolution down-sampled
features. To address such issues, we propose a new approach, namely Temporal
Stencil Modeling (TSM), which combines the strengths of advanced time-series
sequence modeling (with the HiPPO features) and state-of-the-art neural PDE
solvers (with learnable stencil modeling). TSM aims to recover the lost
information from the PDE trajectories and can be regarded as a temporal
generalization of classic finite volume methods such as WENO. Our experimental
results show that TSM achieves the new state-of-the-art simulation accuracy for
2-D incompressible Navier-Stokes turbulent flows: it significantly outperforms
the previously reported best results by 19.9% in terms of the highly-correlated
duration time and reduces the inference latency into 80%. We also show a strong
generalization ability of the proposed method to various out-of-distribution
turbulent flow settings. Our code is available at
"https://github.com/Edward-Sun/TSM-PDE".
Related papers
- Enhanced Spatiotemporal Prediction Using Physical-guided And Frequency-enhanced Recurrent Neural Networks [17.91230192726962]
This paper proposes a physical-guided neural network to estimate the Stemporal dynamics.
We also propose an adaptive second-order Runge-Kutta method with physical constraints to model the physical states more precisely.
Our model outperforms state-of-the-art methods and performs best in datasets, with a much smaller parameter count.
arXiv Detail & Related papers (2024-05-23T12:39:49Z) - Machine-Learned Closure of URANS for Stably Stratified Turbulence: Connecting Physical Timescales & Data Hyperparameters of Deep Time-Series Models [0.0]
We develop time-series machine learning (ML) methods for closure modeling of the Unsteady Reynolds Averaged Navier Stokes equations.
We consider decaying SST which are homogeneous and stably stratified by a uniform density gradient.
We find that the ratio of the timescales of the minimum information required by the ML models to accurately capture the dynamics of the SST corresponds to the Reynolds number of the flow.
arXiv Detail & Related papers (2024-04-24T18:58:00Z) - Multi-fidelity reduced-order surrogate modeling [5.346062841242067]
We present a new data-driven strategy that combines dimensionality reduction with multi-fidelity neural network surrogates.
We show that the onset of instabilities and transients are well captured by this surrogate technique.
arXiv Detail & Related papers (2023-09-01T08:16:53Z) - PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers [40.097474800631]
Time-dependent partial differential equations (PDEs) are ubiquitous in science and engineering.
Deep neural network based surrogates have gained increased interest.
arXiv Detail & Related papers (2023-08-10T17:53:05Z) - Physics-Driven Turbulence Image Restoration with Stochastic Refinement [80.79900297089176]
Image distortion by atmospheric turbulence is a critical problem in long-range optical imaging systems.
Fast and physics-grounded simulation tools have been introduced to help the deep-learning models adapt to real-world turbulence conditions.
This paper proposes the Physics-integrated Restoration Network (PiRN) to help the network to disentangle theity from the degradation and the underlying image.
arXiv Detail & Related papers (2023-07-20T05:49:21Z) - Mixed Effects Neural ODE: A Variational Approximation for Analyzing the
Dynamics of Panel Data [50.23363975709122]
We propose a probabilistic model called ME-NODE to incorporate (fixed + random) mixed effects for analyzing panel data.
We show that our model can be derived using smooth approximations of SDEs provided by the Wong-Zakai theorem.
We then derive Evidence Based Lower Bounds for ME-NODE, and develop (efficient) training algorithms.
arXiv Detail & Related papers (2022-02-18T22:41:51Z) - A Note on Learning Rare Events in Molecular Dynamics using LSTM and
Transformer [4.80427355202687]
Recently successful examples on learning slow dynamics by LSTM are given with simulation data of low dimensional reaction coordinate.
We show that the following three key factors significantly affect the performance of language model learning, namely dimensionality of reaction coordinates, temporal resolution and state partition.
arXiv Detail & Related papers (2021-07-14T09:26:36Z) - Closed-form Continuous-Depth Models [99.40335716948101]
Continuous-depth neural models rely on advanced numerical differential equation solvers.
We present a new family of models, termed Closed-form Continuous-depth (CfC) networks, that are simple to describe and at least one order of magnitude faster.
arXiv Detail & Related papers (2021-06-25T22:08:51Z) - Sparse Flows: Pruning Continuous-depth Models [107.98191032466544]
We show that pruning improves generalization for neural ODEs in generative modeling.
We also show that pruning finds minimal and efficient neural ODE representations with up to 98% less parameters compared to the original network, without loss of accuracy.
arXiv Detail & Related papers (2021-06-24T01:40:17Z) - Neural ODE Processes [64.10282200111983]
We introduce Neural ODE Processes (NDPs), a new class of processes determined by a distribution over Neural ODEs.
We show that our model can successfully capture the dynamics of low-dimensional systems from just a few data-points.
arXiv Detail & Related papers (2021-03-23T09:32:06Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.