Error-in-variables modelling for operator learning

• URL: http://arxiv.org/abs/2204.10909v1
• Date: Fri, 22 Apr 2022 19:54:34 GMT
• Title: Error-in-variables modelling for operator learning
• Authors: Ravi G. Patel, Indu Manickam, Myoungkyu Lee, Mamikon Gulian
• Abstract summary: Failure to account for noisy independent variables can lead to biased parameter estimates. In this work, we derive an analogue of attenuation bias for linear operator regression with white noise in both the independent and dependent variables. We propose error-in-variables (EiV) models for two operator regression methods, MOR-Physics and DeepONet, and demonstrate that these new models reduce bias in the presence of noisy independent variables.
• Score: 0.35880734696551125
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep operator learning has emerged as a promising tool for reduced-order modelling and PDE model discovery. Leveraging the expressive power of deep neural networks, especially in high dimensions, such methods learn the mapping between functional state variables. While proposed methods have assumed noise only in the dependent variables, experimental and numerical data for operator learning typically exhibit noise in the independent variables as well, since both variables represent signals that are subject to measurement error. In regression on scalar data, failure to account for noisy independent variables can lead to biased parameter estimates. With noisy independent variables, linear models fitted via ordinary least squares (OLS) will show attenuation bias, wherein the slope will be underestimated. In this work, we derive an analogue of attenuation bias for linear operator regression with white noise in both the independent and dependent variables. In the nonlinear setting, we computationally demonstrate underprediction of the action of the Burgers operator in the presence of noise in the independent variable. We propose error-in-variables (EiV) models for two operator regression methods, MOR-Physics and DeepONet, and demonstrate that these new models reduce bias in the presence of noisy independent variables for a variety of operator learning problems. Considering the Burgers operator in 1D and 2D, we demonstrate that EiV operator learning robustly recovers operators in high-noise regimes that defeat OLS operator learning. We also introduce an EiV model for time-evolving PDE discovery and show that OLS and EiV perform similarly in learning the Kuramoto-Sivashinsky evolution operator from corrupted data, suggesting that the effect of bias in OLS operator learning depends on the regularity of the target operator.

Related papers

• A Differentiable Partially Observable Generalized Linear Model with Forward-Backward Message Passing [2.600709013150986]
  We propose a new differentiable POGLM, which enables the pathwise gradient estimator, better than the score function gradient estimator used in existing works. Our new method yields more interpretable parameters, underscoring its significance in neuroscience.
  arXiv  Detail & Related papers  (2024-02-02T09:34:49Z)
• Doubly Stochastic Models: Learning with Unbiased Label Noises and Inference Stability [85.1044381834036]
  We investigate the implicit regularization effects of label noises under mini-batch sampling settings of gradient descent. We find such implicit regularizer would favor some convergence points that could stabilize model outputs against perturbation of parameters. Our work doesn't assume SGD as an Ornstein-Uhlenbeck like process and achieve a more general result with convergence of approximation proved.
  arXiv  Detail & Related papers  (2023-04-01T14:09:07Z)
• Sharp Spectral Rates for Koopman Operator Learning [27.820383937933034]
  We present for the first time non-asymptotic learning bounds for the Koopman eigenvalues and eigenfunctions. Our results shed new light on the emergence of spurious eigenvalues.
  arXiv  Detail & Related papers  (2023-02-03T21:19:56Z)
• Operator inference with roll outs for learning reduced models from scarce and low-quality data [0.0]
  We propose to combine data-driven modeling via operator inference with the dynamic training via roll outs of neural ordinary differential equations. We show that operator inference with roll outs provides predictive models from training trajectories even if data are sampled sparsely in time and polluted with noise of up to 10%.
  arXiv  Detail & Related papers  (2022-12-02T19:41:31Z)
• Equivariance and Invariance Inductive Bias for Learning from Insufficient Data [65.42329520528223]
  We show why insufficient data renders the model more easily biased to the limited training environments that are usually different from testing. We propose a class-wise invariant risk minimization (IRM) that efficiently tackles the challenge of missing environmental annotation in conventional IRM.
  arXiv  Detail & Related papers  (2022-07-25T15:26:19Z)
• How robust are pre-trained models to distribution shift? [82.08946007821184]
  We show how spurious correlations affect the performance of popular self-supervised learning (SSL) and auto-encoder based models (AE) We develop a novel evaluation scheme with the linear head trained on out-of-distribution (OOD) data, to isolate the performance of the pre-trained models from a potential bias of the linear head used for evaluation.
  arXiv  Detail & Related papers  (2022-06-17T16:18:28Z)
• Learning Dynamical Systems via Koopman Operator Regression in Reproducing Kernel Hilbert Spaces [52.35063796758121]
  We formalize a framework to learn the Koopman operator from finite data trajectories of the dynamical system. We link the risk with the estimation of the spectral decomposition of the Koopman operator. Our results suggest RRR might be beneficial over other widely used estimators.
  arXiv  Detail & Related papers  (2022-05-27T14:57:48Z)
• Exploring Local Explanations of Nonlinear Models Using Animated Linear Projections [5.524804393257921]
  We show how to use eXplainable AI (XAI) to shed light on how a model use predictors to arrive at a prediction. To understand how the interaction between predictors affects the variable importance estimate, we can convert LVAs into linear projections. The approach is illustrated with examples from categorical (penguin species, chocolate types) and quantitative (soccer/football salaries, house prices) response models.
  arXiv  Detail & Related papers  (2022-05-11T09:11:02Z)
• Equivariance Allows Handling Multiple Nuisance Variables When Analyzing Pooled Neuroimaging Datasets [53.34152466646884]
  In this paper, we show how bringing recent results on equivariant representation learning instantiated on structured spaces together with simple use of classical results on causal inference provides an effective practical solution. We demonstrate how our model allows dealing with more than one nuisance variable under some assumptions and can enable analysis of pooled scientific datasets in scenarios that would otherwise entail removing a large portion of the samples.
  arXiv  Detail & Related papers  (2022-03-29T04:54:06Z)
• Bias-Variance Tradeoffs in Single-Sample Binary Gradient Estimators [100.58924375509659]
  Straight-through (ST) estimator gained popularity due to its simplicity and efficiency. Several techniques were proposed to improve over ST while keeping the same low computational complexity. We conduct a theoretical analysis of Bias and Variance of these methods in order to understand tradeoffs and verify originally claimed properties.
  arXiv  Detail & Related papers  (2021-10-07T15:16:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.