Understanding Generalization in Physics Informed Models through Affine Variety Dimensions

• URL: http://arxiv.org/abs/2501.18879v1
• Date: Fri, 31 Jan 2025 04:25:22 GMT
• Title: Understanding Generalization in Physics Informed Models through Affine Variety Dimensions
• Authors: Takeshi Koshizuka, Issei Sato,
• Abstract summary: We show that the generalization performance of linear regressors incorporating differential equation structures is determined by the dimension of the associated affine variety. This finding enables a unified analysis of various equations, including nonlinear ones.
• Score: 35.17568416175663
• License:
• Abstract: In recent years, physics-informed machine learning has gained significant attention for its ability to enhance statistical performance and sample efficiency by integrating physical structures into machine learning models. These structures, such as differential equations, conservation laws, and symmetries, serve as inductive biases that can improve the generalization capacity of the hybrid model. However, the mechanisms by which these physical structures enhance generalization capacity are not fully understood, limiting the ability to guarantee the performance of the models. In this study, we show that the generalization performance of linear regressors incorporating differential equation structures is determined by the dimension of the associated affine variety, rather than the number of parameters. This finding enables a unified analysis of various equations, including nonlinear ones. We introduce a method to approximate the dimension of the affine variety and provide experimental evidence to validate our theoretical insights.

Related papers

• Geometric Understanding of Discriminability and Transferability for Visual Domain Adaptation [27.326817457760725]
  Invariant representation learning for unsupervised domain adaptation (UDA) has made significant advances in computer vision and pattern recognition communities. Recently, empirical connections between transferability and discriminability have received increasing attention. In this work, we systematically analyze the essentials of transferability and discriminability from the geometric perspective.
  arXiv  Detail & Related papers  (2024-06-24T13:31:08Z)
• Assessment of Uncertainty Quantification in Universal Differential Equations [1.374796982212312]
  Universal Differential Equations (UDEs) are used to combine prior knowledge in the form of mechanistic formulations with universal function approximators, like neural networks. We provide a formalisation of uncertainty quantification (UQ) for UDEs and investigate important frequentist and Bayesian methods.
  arXiv  Detail & Related papers  (2024-06-13T06:36:19Z)
• Scaling and renormalization in high-dimensional regression [72.59731158970894]
  This paper presents a succinct derivation of the training and generalization performance of a variety of high-dimensional ridge regression models. We provide an introduction and review of recent results on these topics, aimed at readers with backgrounds in physics and deep learning.
  arXiv  Detail & Related papers  (2024-05-01T15:59:00Z)
• Shape Arithmetic Expressions: Advancing Scientific Discovery Beyond Closed-Form Equations [56.78271181959529]
  Generalized Additive Models (GAMs) can capture non-linear relationships between variables and targets, but they cannot capture intricate feature interactions. We propose Shape Expressions Arithmetic ( SHAREs) that fuses GAM's flexible shape functions with the complex feature interactions found in mathematical expressions. We also design a set of rules for constructing SHAREs that guarantee transparency of the found expressions beyond the standard constraints.
  arXiv  Detail & Related papers  (2024-04-15T13:44:01Z)
• Discovering Interpretable Physical Models using Symbolic Regression and Discrete Exterior Calculus [55.2480439325792]
  We propose a framework that combines Symbolic Regression (SR) and Discrete Exterior Calculus (DEC) for the automated discovery of physical models. DEC provides building blocks for the discrete analogue of field theories, which are beyond the state-of-the-art applications of SR to physical problems. We prove the effectiveness of our methodology by re-discovering three models of Continuum Physics from synthetic experimental data.
  arXiv  Detail & Related papers  (2023-10-10T13:23:05Z)
• Symmetry Group Equivariant Architectures for Physics [52.784926970374556]
  In the domain of machine learning, an awareness of symmetries has driven impressive performance breakthroughs. We argue that both the physics community and the broader machine learning community have much to understand.
  arXiv  Detail & Related papers  (2022-03-11T18:27:04Z)
• Equivariant vector field network for many-body system modeling [65.22203086172019]
  Equivariant Vector Field Network (EVFN) is built on a novel equivariant basis and the associated scalarization and vectorization layers. We evaluate our method on predicting trajectories of simulated Newton mechanics systems with both full and partially observed data.
  arXiv  Detail & Related papers  (2021-10-26T14:26:25Z)
• Approximate Latent Force Model Inference [1.3927943269211591]
  latent force models offer an interpretable alternative to purely data driven tools for inference in dynamical systems. We show that a neural operator approach can scale our model to thousands of instances, enabling fast, distributed computation.
  arXiv  Detail & Related papers  (2021-09-24T09:55:00Z)
• ICE-BeeM: Identifiable Conditional Energy-Based Deep Models Based on Nonlinear ICA [11.919315372249802]
  We consider the identifiability theory of probabilistic models. We show that our model can be used for the estimation of the components in the framework of Independently Modulated Component Analysis.
  arXiv  Detail & Related papers  (2020-02-26T14:43:30Z)
• Geometric deep learning for computational mechanics Part I: Anisotropic Hyperelasticity [1.8606313462183062]
  This paper is the first attempt to use geometric deep learning and Sobolev training incorporate non-Euclidean microstructural data such that anisotropic hyperstructural material machine learning models can be trained in the finite deformation range.
  arXiv  Detail & Related papers  (2020-01-08T02:07:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.