Parsimony-Enhanced Sparse Bayesian Learning for Robust Discovery of
Partial Differential Equations
- URL: http://arxiv.org/abs/2107.07040v1
- Date: Thu, 8 Jul 2021 00:56:11 GMT
- Title: Parsimony-Enhanced Sparse Bayesian Learning for Robust Discovery of
Partial Differential Equations
- Authors: Zhiming Zhang and Yongming Liu
- Abstract summary: A Parsimony Enhanced Sparse Bayesian Learning (PeSBL) method is developed for discovering the governing Partial Differential Equations (PDEs) of nonlinear dynamical systems.
Results of numerical case studies indicate that the governing PDEs of many canonical dynamical systems can be correctly identified using the proposed PeSBL method.
- Score: 5.584060970507507
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Robust physics discovery is of great interest for many scientific and
engineering fields. Inspired by the principle that a representative model is
the one simplest possible, a new model selection criteria considering both
model's Parsimony and Sparsity is proposed. A Parsimony Enhanced Sparse
Bayesian Learning (PeSBL) method is developed for discovering the governing
Partial Differential Equations (PDEs) of nonlinear dynamical systems. Compared
with the conventional Sparse Bayesian Learning (SBL) method, the PeSBL method
promotes parsimony of the learned model in addition to its sparsity. In this
method, the parsimony of model terms is evaluated using their locations in the
prescribed candidate library, for the first time, considering the increased
complexity with the power of polynomials and the order of spatial derivatives.
Subsequently, the model parameters are updated through Bayesian inference with
the raw data. This procedure aims to reduce the error associated with the
possible loss of information in data preprocessing and numerical
differentiation prior to sparse regression. Results of numerical case studies
indicate that the governing PDEs of many canonical dynamical systems can be
correctly identified using the proposed PeSBL method from highly noisy data (up
to 50% in the current study). Next, the proposed methodology is extended for
stochastic PDE learning where all parameters and modeling error are considered
as random variables. Hierarchical Bayesian Inference (HBI) is integrated with
the proposed framework for stochastic PDE learning from a population of
observations. Finally, the proposed PeSBL is demonstrated for system response
prediction with uncertainties and anomaly diagnosis. Codes of all demonstrated
examples in this study are available on the website: https://github.com/ymlasu.
Related papers
- Leveraging viscous Hamilton-Jacobi PDEs for uncertainty quantification in scientific machine learning [1.8175282137722093]
Uncertainty (UQ) in scientific machine learning (SciML) combines the powerful predictive power of SciML with methods for quantifying the reliability of the learned models.
We provide a new interpretation for UQ problems by establishing a new theoretical connection between some Bayesian inference problems arising in SciML and viscous Hamilton-Jacobi partial differential equations (HJ PDEs)
We develop a new Riccati-based methodology that provides computational advantages when continuously updating the model predictions.
arXiv Detail & Related papers (2024-04-12T20:54:01Z) - Diffusion posterior sampling for simulation-based inference in tall data settings [53.17563688225137]
Simulation-based inference ( SBI) is capable of approximating the posterior distribution that relates input parameters to a given observation.
In this work, we consider a tall data extension in which multiple observations are available to better infer the parameters of the model.
We compare our method to recently proposed competing approaches on various numerical experiments and demonstrate its superiority in terms of numerical stability and computational cost.
arXiv Detail & Related papers (2024-04-11T09:23:36Z) - Reflected Diffusion Models [93.26107023470979]
We present Reflected Diffusion Models, which reverse a reflected differential equation evolving on the support of the data.
Our approach learns the score function through a generalized score matching loss and extends key components of standard diffusion models.
arXiv Detail & Related papers (2023-04-10T17:54:38Z) - Deep Learning Aided Laplace Based Bayesian Inference for Epidemiological
Systems [2.596903831934905]
We propose a hybrid approach where Laplace-based Bayesian inference is combined with an ANN architecture for obtaining approximations to the ODE trajectories.
The effectiveness of our proposed methods is demonstrated using an epidemiological system with non-analytical solutions, the Susceptible-Infectious-Removed (SIR) model for infectious diseases.
arXiv Detail & Related papers (2022-10-17T09:02:41Z) - Mixed Effects Neural ODE: A Variational Approximation for Analyzing the
Dynamics of Panel Data [50.23363975709122]
We propose a probabilistic model called ME-NODE to incorporate (fixed + random) mixed effects for analyzing panel data.
We show that our model can be derived using smooth approximations of SDEs provided by the Wong-Zakai theorem.
We then derive Evidence Based Lower Bounds for ME-NODE, and develop (efficient) training algorithms.
arXiv Detail & Related papers (2022-02-18T22:41:51Z) - Improving Robustness and Uncertainty Modelling in Neural Ordinary
Differential Equations [0.2538209532048866]
We propose a novel approach to model uncertainty in NODE by considering a distribution over the end-time $T$ of the ODE solver.
We also propose, adaptive latent time NODE (ALT-NODE), which allow each data point to have a distinct posterior distribution over end-times.
We demonstrate the effectiveness of the proposed approaches in modelling uncertainty and robustness through experiments on synthetic and several real-world image classification data.
arXiv Detail & Related papers (2021-12-23T16:56:10Z) - Inverting brain grey matter models with likelihood-free inference: a
tool for trustable cytoarchitecture measurements [62.997667081978825]
characterisation of the brain grey matter cytoarchitecture with quantitative sensitivity to soma density and volume remains an unsolved challenge in dMRI.
We propose a new forward model, specifically a new system of equations, requiring a few relatively sparse b-shells.
We then apply modern tools from Bayesian analysis known as likelihood-free inference (LFI) to invert our proposed model.
arXiv Detail & Related papers (2021-11-15T09:08:27Z) - MINIMALIST: Mutual INformatIon Maximization for Amortized Likelihood
Inference from Sampled Trajectories [61.3299263929289]
Simulation-based inference enables learning the parameters of a model even when its likelihood cannot be computed in practice.
One class of methods uses data simulated with different parameters to infer an amortized estimator for the likelihood-to-evidence ratio.
We show that this approach can be formulated in terms of mutual information between model parameters and simulated data.
arXiv Detail & Related papers (2021-06-03T12:59:16Z) - Gaussian processes meet NeuralODEs: A Bayesian framework for learning
the dynamics of partially observed systems from scarce and noisy data [0.0]
This paper presents a machine learning framework (GP-NODE) for Bayesian systems identification from partial, noisy and irregular observations of nonlinear dynamical systems.
The proposed method takes advantage of recent developments in differentiable programming to propagate gradient information through ordinary differential equation solvers.
A series of numerical studies is presented to demonstrate the effectiveness of the proposed GP-NODE method including predator-prey systems, systems biology, and a 50-dimensional human motion dynamical system.
arXiv Detail & Related papers (2021-03-04T23:42:14Z) - Bayesian data-driven discovery of partial differential equations with variable coefficients [9.331440154110117]
We propose an advanced Bayesian sparse learning algorithm for PDE discovery with variable coefficients.
In the experiments, we show that the tBGL-SS method is more robust than the baseline methods under noisy environments.
arXiv Detail & Related papers (2021-02-02T11:05:34Z) - Probabilistic Circuits for Variational Inference in Discrete Graphical
Models [101.28528515775842]
Inference in discrete graphical models with variational methods is difficult.
Many sampling-based methods have been proposed for estimating Evidence Lower Bound (ELBO)
We propose a new approach that leverages the tractability of probabilistic circuit models, such as Sum Product Networks (SPN)
We show that selective-SPNs are suitable as an expressive variational distribution, and prove that when the log-density of the target model is aweighted the corresponding ELBO can be computed analytically.
arXiv Detail & Related papers (2020-10-22T05:04:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.