Optimal Linear Baseline Models for Scientific Machine Learning

• URL: http://arxiv.org/abs/2508.05831v1
• Date: Thu, 07 Aug 2025 20:17:07 GMT
• Title: Optimal Linear Baseline Models for Scientific Machine Learning
• Authors: Alexander DeLise, Kyle Loh, Krish Patel, Meredith Teague, Andrea Arnold, Matthias Chung,
• Abstract summary: We develop a unified theoretical framework for analyzing linear encoder-decoder architectures.<n>We derive closed-form, rank-constrained linear optimal mappings for forward modeling and inverse recovery tasks.<n>This work provides a robust baseline for understanding and benchmarking learned neural network models for scientific machine learning problems.
• Score: 37.69303106863453
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Across scientific domains, a fundamental challenge is to characterize and compute the mappings from underlying physical processes to observed signals and measurements. While nonlinear neural networks have achieved considerable success, they remain theoretically opaque, which hinders adoption in contexts where interpretability is paramount. In contrast, linear neural networks serve as a simple yet effective foundation for gaining insight into these complex relationships. In this work, we develop a unified theoretical framework for analyzing linear encoder-decoder architectures through the lens of Bayes risk minimization for solving data-driven scientific machine learning problems. We derive closed-form, rank-constrained linear and affine linear optimal mappings for forward modeling and inverse recovery tasks. Our results generalize existing formulations by accommodating rank-deficiencies in data, forward operators, and measurement processes. We validate our theoretical results by conducting numerical experiments on datasets from simple biomedical imaging, financial factor analysis, and simulations involving nonlinear fluid dynamics via the shallow water equations. This work provides a robust baseline for understanding and benchmarking learned neural network models for scientific machine learning problems.

Related papers

• Fusing CFD and measurement data using transfer learning [49.1574468325115]
  We introduce a non-linear method based on neural networks combining simulation and measurement data via transfer learning.<n>In a first step, the neural network is trained on simulation data to learn spatial features of the distributed quantities.<n>The second step involves transfer learning on the measurement data to correct for systematic errors between simulation and measurement by only re-training a small subset of the entire neural network model.
  arXiv  Detail & Related papers  (2025-07-28T07:21:46Z)
• Physics Informed Constrained Learning of Dynamics from Static Data [8.346864633675414]
  A physics-informed neural network (PINN) models the dynamics of a system by integrating the governing physical laws into the architecture of a neural network.<n>Existing PINN frameworks rely on fully observed time-course data, the acquisition of which could be prohibitive for many systems.<n>In this study, we developed a new PINN learning paradigm, namely Constrained Learning, that enables the approximation of first-order derivatives or motions using non-time course or partially observed data.
  arXiv  Detail & Related papers  (2025-04-17T06:06:53Z)
• Multiscale Analysis of Woven Composites Using Hierarchical Physically Recurrent Neural Networks [0.0]
  Multiscale homogenization of woven composites requires detailed micromechanical evaluations.<n>This study introduces a Hierarchical Physically Recurrent Neural Network (HPRNN) employing two levels of surrogate modeling.
  arXiv  Detail & Related papers  (2025-03-06T19:02:32Z)
• Generalized Factor Neural Network Model for High-dimensional Regression [50.554377879576066]
  We tackle the challenges of modeling high-dimensional data sets with latent low-dimensional structures hidden within complex, non-linear, and noisy relationships.<n>Our approach enables a seamless integration of concepts from non-parametric regression, factor models, and neural networks for high-dimensional regression.
  arXiv  Detail & Related papers  (2025-02-16T23:13:55Z)
• Deep Learning Through A Telescoping Lens: A Simple Model Provides Empirical Insights On Grokking, Gradient Boosting & Beyond [61.18736646013446]
  In pursuit of a deeper understanding of its surprising behaviors, we investigate the utility of a simple yet accurate model of a trained neural network. Across three case studies, we illustrate how it can be applied to derive new empirical insights on a diverse range of prominent phenomena.
  arXiv  Detail & Related papers  (2024-10-31T22:54:34Z)
• Physics-Informed Machine Learning for Seismic Response Prediction OF Nonlinear Steel Moment Resisting Frame Structures [6.483318568088176]
  PiML method integrates scientific principles and physical laws into deep neural networks to model seismic responses of nonlinear structures. Manipulating the equation of motion helps learn system nonlinearities and confines solutions within physically interpretable results. Result handles complex data better than existing physics-guided LSTM models and outperforms other non-physics data-driven networks.
  arXiv  Detail & Related papers  (2024-02-28T02:16:03Z)
• Mechanistic Neural Networks for Scientific Machine Learning [58.99592521721158]
  We present Mechanistic Neural Networks, a neural network design for machine learning applications in the sciences. It incorporates a new Mechanistic Block in standard architectures to explicitly learn governing differential equations as representations. Central to our approach is a novel Relaxed Linear Programming solver (NeuRLP) inspired by a technique that reduces solving linear ODEs to solving linear programs.
  arXiv  Detail & Related papers  (2024-02-20T15:23:24Z)
• Understanding Multi-phase Optimization Dynamics and Rich Nonlinear Behaviors of ReLU Networks [8.180184504355571]
  We conduct a theoretical characterization of the training process of two-layer ReLU network trained by Gradient Flow on a linearlyparable data. We reveal four different phases from the whole training process showing a general simplifying-to-complicating learning trend. Specific nonlinear behaviors can also be precisely identified captured theoretically, such as initial, saddle-plateau dynamics, condensation escape, changes of activation patterns with increasing complexity.
  arXiv  Detail & Related papers  (2023-05-21T14:08:34Z)
• Persistence-based operators in machine learning [62.997667081978825]
  We introduce a class of persistence-based neural network layers. Persistence-based layers allow the users to easily inject knowledge about symmetries respected by the data, are equipped with learnable weights, and can be composed with state-of-the-art neural architectures.
  arXiv  Detail & Related papers  (2022-12-28T18:03:41Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.