Adaptive importance sampling for Deep Ritz
- URL: http://arxiv.org/abs/2310.17185v2
- Date: Mon, 30 Oct 2023 11:42:29 GMT
- Title: Adaptive importance sampling for Deep Ritz
- Authors: Xiaoliang Wan and Tao Zhou and Yuancheng Zhou
- Abstract summary: We introduce an adaptive sampling method for the Deep Ritz method aimed at solving partial differential equations (PDEs)
One network is employed to approximate the solution of PDEs, while the other one is a deep generative model used to generate new collocation points to refine the training set.
Compared to the original Deep Ritz method, the proposed adaptive method improves accuracy, especially for problems characterized by low regularity and high dimensionality.
- Score: 7.123920027048777
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We introduce an adaptive sampling method for the Deep Ritz method aimed at
solving partial differential equations (PDEs). Two deep neural networks are
used. One network is employed to approximate the solution of PDEs, while the
other one is a deep generative model used to generate new collocation points to
refine the training set. The adaptive sampling procedure consists of two main
steps. The first step is solving the PDEs using the Deep Ritz method by
minimizing an associated variational loss discretized by the collocation points
in the training set. The second step involves generating a new training set,
which is then used in subsequent computations to further improve the accuracy
of the current approximate solution. We treat the integrand in the variational
loss as an unnormalized probability density function (PDF) and approximate it
using a deep generative model called bounded KRnet. The new samples and their
associated PDF values are obtained from the bounded KRnet. With these new
samples and their associated PDF values, the variational loss can be
approximated more accurately by importance sampling. Compared to the original
Deep Ritz method, the proposed adaptive method improves accuracy, especially
for problems characterized by low regularity and high dimensionality. We
demonstrate the effectiveness of our new method through a series of numerical
experiments.
Related papers
- Total Uncertainty Quantification in Inverse PDE Solutions Obtained with Reduced-Order Deep Learning Surrogate Models [50.90868087591973]
We propose an approximate Bayesian method for quantifying the total uncertainty in inverse PDE solutions obtained with machine learning surrogate models.
We test the proposed framework by comparing it with the iterative ensemble smoother and deep ensembling methods for a non-linear diffusion equation.
arXiv Detail & Related papers (2024-08-20T19:06:02Z) - Deep adaptive sampling for surrogate modeling without labeled data [4.047684532081032]
We present a deep adaptive sampling method for surrogate modeling ($textDAS2$)
In the parametric setting, the residual loss function can be regarded as an unnormalized probability density function.
New samples match the residual-induced distribution, the refined training set can further reduce the statistical error.
arXiv Detail & Related papers (2024-02-17T13:44:02Z) - Variational Density Propagation Continual Learning [0.0]
Deep Neural Networks (DNNs) deployed to the real world are regularly subject to out-of-distribution (OoD) data.
This paper proposes a framework for adapting to data distribution drift modeled by benchmark Continual Learning datasets.
arXiv Detail & Related papers (2023-08-22T21:51:39Z) - Low-rank extended Kalman filtering for online learning of neural
networks from streaming data [71.97861600347959]
We propose an efficient online approximate Bayesian inference algorithm for estimating the parameters of a nonlinear function from a potentially non-stationary data stream.
The method is based on the extended Kalman filter (EKF), but uses a novel low-rank plus diagonal decomposition of the posterior matrix.
In contrast to methods based on variational inference, our method is fully deterministic, and does not require step-size tuning.
arXiv Detail & Related papers (2023-05-31T03:48:49Z) - Adversarial Adaptive Sampling: Unify PINN and Optimal Transport for the Approximation of PDEs [2.526490864645154]
We propose a new minmax formulation to optimize simultaneously the approximate solution, given by a neural network model, and the random samples in the training set.
The key idea is to use a deep generative model to adjust random samples in the training set such that the residual induced by the approximate PDE solution can maintain a smooth profile.
arXiv Detail & Related papers (2023-05-30T02:59:18Z) - A DeepParticle method for learning and generating aggregation patterns
in multi-dimensional Keller-Segel chemotaxis systems [3.6184545598911724]
We study a regularized interacting particle method for computing aggregation patterns and near singular solutions of a Keller-Segal (KS) chemotaxis system in two and three space dimensions.
We further develop DeepParticle (DP) method to learn and generate solutions under variations of physical parameters.
arXiv Detail & Related papers (2022-08-31T20:52:01Z) - An application of the splitting-up method for the computation of a
neural network representation for the solution for the filtering equations [68.8204255655161]
Filtering equations play a central role in many real-life applications, including numerical weather prediction, finance and engineering.
One of the classical approaches to approximate the solution of the filtering equations is to use a PDE inspired method, called the splitting-up method.
We combine this method with a neural network representation to produce an approximation of the unnormalised conditional distribution of the signal process.
arXiv Detail & Related papers (2022-01-10T11:01:36Z) - DAS: A deep adaptive sampling method for solving partial differential
equations [2.934397685379054]
We propose a deep adaptive sampling (DAS) method for solving partial differential equations (PDEs)
Deep neural networks are utilized to approximate the solutions of PDEs and deep generative models are employed to generate new collocation points that refine the training set.
We present a theoretical analysis to show that the proposed DAS method can reduce the error bound and demonstrate its effectiveness with numerical experiments.
arXiv Detail & Related papers (2021-12-28T08:37:47Z) - Learning Sampling Policy for Faster Derivative Free Optimization [100.27518340593284]
We propose a new reinforcement learning based ZO algorithm (ZO-RL) with learning the sampling policy for generating the perturbations in ZO optimization instead of using random sampling.
Our results show that our ZO-RL algorithm can effectively reduce the variances of ZO gradient by learning a sampling policy, and converge faster than existing ZO algorithms in different scenarios.
arXiv Detail & Related papers (2021-04-09T14:50:59Z) - Path Sample-Analytic Gradient Estimators for Stochastic Binary Networks [78.76880041670904]
In neural networks with binary activations and or binary weights the training by gradient descent is complicated.
We propose a new method for this estimation problem combining sampling and analytic approximation steps.
We experimentally show higher accuracy in gradient estimation and demonstrate a more stable and better performing training in deep convolutional models.
arXiv Detail & Related papers (2020-06-04T21:51:21Z) - Neural Control Variates [71.42768823631918]
We show that a set of neural networks can face the challenge of finding a good approximation of the integrand.
We derive a theoretically optimal, variance-minimizing loss function, and propose an alternative, composite loss for stable online training in practice.
Specifically, we show that the learned light-field approximation is of sufficient quality for high-order bounces, allowing us to omit the error correction and thereby dramatically reduce the noise at the cost of negligible visible bias.
arXiv Detail & Related papers (2020-06-02T11:17:55Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.