Generalization Error Guaranteed Auto-Encoder-Based Nonlinear Model
Reduction for Operator Learning
- URL: http://arxiv.org/abs/2401.10490v1
- Date: Fri, 19 Jan 2024 05:01:43 GMT
- Title: Generalization Error Guaranteed Auto-Encoder-Based Nonlinear Model
Reduction for Operator Learning
- Authors: Hao Liu, Biraj Dahal, Rongjie Lai, Wenjing Liao
- Abstract summary: In this paper, we utilize low-dimensional nonlinear structures in model reduction by investigating Auto-Encoder-based Neural Network (AENet)
Our numerical experiments validate the ability of AENet to accurately learn the solution operator of nonlinear partial differential equations.
Our theoretical framework shows that the sample complexity of training AENet is intricately tied to the intrinsic dimension of the modeled process.
- Score: 12.124206935054389
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Many physical processes in science and engineering are naturally represented
by operators between infinite-dimensional function spaces. The problem of
operator learning, in this context, seeks to extract these physical processes
from empirical data, which is challenging due to the infinite or high
dimensionality of data. An integral component in addressing this challenge is
model reduction, which reduces both the data dimensionality and problem size.
In this paper, we utilize low-dimensional nonlinear structures in model
reduction by investigating Auto-Encoder-based Neural Network (AENet). AENet
first learns the latent variables of the input data and then learns the
transformation from these latent variables to corresponding output data. Our
numerical experiments validate the ability of AENet to accurately learn the
solution operator of nonlinear partial differential equations. Furthermore, we
establish a mathematical and statistical estimation theory that analyzes the
generalization error of AENet. Our theoretical framework shows that the sample
complexity of training AENet is intricately tied to the intrinsic dimension of
the modeled process, while also demonstrating the remarkable resilience of
AENet to noise.
Related papers
- DimOL: Dimensional Awareness as A New 'Dimension' in Operator Learning [63.5925701087252]
We introduce DimOL (Dimension-aware Operator Learning), drawing insights from dimensional analysis.
To implement DimOL, we propose the ProdLayer, which can be seamlessly integrated into FNO-based and Transformer-based PDE solvers.
Empirically, DimOL models achieve up to 48% performance gain within the PDE datasets.
arXiv Detail & Related papers (2024-10-08T10:48:50Z) - Hybrid data-driven and physics-informed regularized learning of cyclic
plasticity with Neural Networks [0.0]
The proposed model architecture is simpler and more efficient compared to existing solutions from the literature.
The validation of the approach is carried out by means of surrogate data obtained with the Armstrong-Frederick kinematic hardening model.
arXiv Detail & Related papers (2024-03-04T07:09:54Z) - Peridynamic Neural Operators: A Data-Driven Nonlocal Constitutive Model
for Complex Material Responses [12.454290779121383]
We introduce a novel integral neural operator architecture called the Peridynamic Neural Operator (PNO) that learns a nonlocal law from data.
This neural operator provides a forward model in the form of state-based peridynamics, with objectivity and momentum balance laws automatically guaranteed.
We show that, owing to its ability to capture complex responses, our learned neural operator achieves improved accuracy and efficiency compared to baseline models.
arXiv Detail & Related papers (2024-01-11T17:37:20Z) - Discovering Interpretable Physical Models using Symbolic Regression and
Discrete Exterior Calculus [55.2480439325792]
We propose a framework that combines Symbolic Regression (SR) and Discrete Exterior Calculus (DEC) for the automated discovery of physical models.
DEC provides building blocks for the discrete analogue of field theories, which are beyond the state-of-the-art applications of SR to physical problems.
We prove the effectiveness of our methodology by re-discovering three models of Continuum Physics from synthetic experimental data.
arXiv Detail & Related papers (2023-10-10T13:23:05Z) - Capturing dynamical correlations using implicit neural representations [85.66456606776552]
We develop an artificial intelligence framework which combines a neural network trained to mimic simulated data from a model Hamiltonian with automatic differentiation to recover unknown parameters from experimental data.
In doing so, we illustrate the ability to build and train a differentiable model only once, which then can be applied in real-time to multi-dimensional scattering data.
arXiv Detail & Related papers (2023-04-08T07:55:36Z) - On Robust Numerical Solver for ODE via Self-Attention Mechanism [82.95493796476767]
We explore training efficient and robust AI-enhanced numerical solvers with a small data size by mitigating intrinsic noise disturbances.
We first analyze the ability of the self-attention mechanism to regulate noise in supervised learning and then propose a simple-yet-effective numerical solver, Attr, which introduces an additive self-attention mechanism to the numerical solution of differential equations.
arXiv Detail & Related papers (2023-02-05T01:39:21Z) - INO: Invariant Neural Operators for Learning Complex Physical Systems
with Momentum Conservation [8.218875461185016]
We introduce a novel integral neural operator architecture, to learn physical models with fundamental conservation laws automatically guaranteed.
As applications, we demonstrate the expressivity and efficacy of our model in learning complex material behaviors from both synthetic and experimental datasets.
arXiv Detail & Related papers (2022-12-29T16:40:41Z) - Physics-guided Data Augmentation for Learning the Solution Operator of
Linear Differential Equations [2.1850269949775663]
We propose a physics-guided data augmentation (PGDA) method to improve the accuracy and generalization of neural operator models.
We demonstrate the advantage of PGDA on a variety of linear differential equations, showing that PGDA can improve the sample complexity and is robust to distributional shift.
arXiv Detail & Related papers (2022-12-08T06:29:15Z) - Learning Deep Implicit Fourier Neural Operators (IFNOs) with
Applications to Heterogeneous Material Modeling [3.9181541460605116]
We propose to use data-driven modeling to predict a material's response without using conventional models.
The material response is modeled by learning the implicit mappings between loading conditions and the resultant displacement and/or damage fields.
We demonstrate the performance of our proposed method for a number of examples, including hyperelastic, anisotropic and brittle materials.
arXiv Detail & Related papers (2022-03-15T19:08:13Z) - Equivariant vector field network for many-body system modeling [65.22203086172019]
Equivariant Vector Field Network (EVFN) is built on a novel equivariant basis and the associated scalarization and vectorization layers.
We evaluate our method on predicting trajectories of simulated Newton mechanics systems with both full and partially observed data.
arXiv Detail & Related papers (2021-10-26T14:26:25Z) - Rank-R FNN: A Tensor-Based Learning Model for High-Order Data
Classification [69.26747803963907]
Rank-R Feedforward Neural Network (FNN) is a tensor-based nonlinear learning model that imposes Canonical/Polyadic decomposition on its parameters.
First, it handles inputs as multilinear arrays, bypassing the need for vectorization, and can thus fully exploit the structural information along every data dimension.
We establish the universal approximation and learnability properties of Rank-R FNN, and we validate its performance on real-world hyperspectral datasets.
arXiv Detail & Related papers (2021-04-11T16:37:32Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.