Orthogonal projection-based regularization for efficient model augmentation

• URL: http://arxiv.org/abs/2501.05842v1
• Date: Fri, 10 Jan 2025 10:33:13 GMT
• Title: Orthogonal projection-based regularization for efficient model augmentation
• Authors: Bendegúz M. Györök, Jan H. Hoekstra, Johan Kon, Tamás Péni, Maarten Schoukens, Roland Tóth,
• Abstract summary: Deep-learning-based nonlinear system identification has shown the ability to produce reliable and highly accurate models in practice. Black-box models lack physical interpretability, and often a considerable part of the learning effort is spent on capturing already expected/known behavior. A potential solution is to integrate prior physical knowledge directly into the model structure, combining the strengths of physics-based modeling and deep-learning identification.
• Score: 2.6071013155805556
• License:
• Abstract: Deep-learning-based nonlinear system identification has shown the ability to produce reliable and highly accurate models in practice. However, these black-box models lack physical interpretability, and often a considerable part of the learning effort is spent on capturing already expected/known behavior due to first-principles-based understanding of some aspects of the system. A potential solution is to integrate prior physical knowledge directly into the model structure, combining the strengths of physics-based modeling and deep-learning-based identification. The most common approach is to use an additive model augmentation structure, where the physics-based and the machine-learning (ML) components are connected in parallel. However, such models are overparametrized, training them is challenging, potentially causing the physics-based part to lose interpretability. To overcome this challenge, this paper proposes an orthogonal projection-based regularization technique to enhance parameter learning, convergence, and even model accuracy in learning-based augmentation of nonlinear baseline models.

Related papers

• No Equations Needed: Learning System Dynamics Without Relying on Closed-Form ODEs [56.78271181959529]
  This paper proposes a conceptual shift to modeling low-dimensional dynamical systems by departing from the traditional two-step modeling process. Instead of first discovering a closed-form equation and then analyzing it, our approach, direct semantic modeling, predicts the semantic representation of the dynamical system. Our approach not only simplifies the modeling pipeline but also enhances the transparency and flexibility of the resulting models.
  arXiv  Detail & Related papers  (2025-01-30T18:36:48Z)
• Enhancing Dynamical System Modeling through Interpretable Machine Learning Augmentations: A Case Study in Cathodic Electrophoretic Deposition [0.8796261172196743]
  We introduce a comprehensive data-driven framework aimed at enhancing the modeling of physical systems. As a demonstrative application, we pursue the modeling of cathodic electrophoretic deposition (EPD), commonly known as e-coating.
  arXiv  Detail & Related papers  (2024-01-16T14:58:21Z)
• Online Calibration of Deep Learning Sub-Models for Hybrid Numerical Modeling Systems [34.50407690251862]
  We present an efficient and practical online learning approach for hybrid systems. We demonstrate that the method, called EGA for Euler Gradient Approximation, converges to the exact gradients in the limit of infinitely small time steps. Results show significant improvements over offline learning, highlighting the potential of end-to-end online learning for hybrid modeling.
  arXiv  Detail & Related papers  (2023-11-17T17:36:26Z)
• Multi-Objective Physics-Guided Recurrent Neural Networks for Identifying Non-Autonomous Dynamical Systems [0.0]
  We propose a physics-guided hybrid approach for modeling non-autonomous systems under control. This is extended by a recurrent neural network and trained using a sophisticated multi-objective strategy. Experiments conducted on real data reveal substantial accuracy improvements by our approach compared to a physics-based model.
  arXiv  Detail & Related papers  (2022-04-27T14:33:02Z)
• Learning continuous models for continuous physics [94.42705784823997]
  We develop a test based on numerical analysis theory to validate machine learning models for science and engineering applications. Our results illustrate how principled numerical analysis methods can be coupled with existing ML training/testing methodologies to validate models for science and engineering applications.
  arXiv  Detail & Related papers  (2022-02-17T07:56:46Z)
• Leveraging the structure of dynamical systems for data-driven modeling [111.45324708884813]
  We consider the impact of the training set and its structure on the quality of the long-term prediction. We show how an informed design of the training set, based on invariants of the system and the structure of the underlying attractor, significantly improves the resulting models.
  arXiv  Detail & Related papers  (2021-12-15T20:09:20Z)
• Which priors matter? Benchmarking models for learning latent dynamics [70.88999063639146]
  Several methods have proposed to integrate priors from classical mechanics into machine learning models. We take a sober look at the current capabilities of these models. We find that the use of continuous and time-reversible dynamics benefits models of all classes.
  arXiv  Detail & Related papers  (2021-11-09T23:48:21Z)
• Constructing Neural Network-Based Models for Simulating Dynamical Systems [59.0861954179401]
  Data-driven modeling is an alternative paradigm that seeks to learn an approximation of the dynamics of a system using observations of the true system. This paper provides a survey of the different ways to construct models of dynamical systems using neural networks. In addition to the basic overview, we review the related literature and outline the most significant challenges from numerical simulations that this modeling paradigm must overcome.
  arXiv  Detail & Related papers  (2021-11-02T10:51:42Z)
• Physics-guided Deep Markov Models for Learning Nonlinear Dynamical Systems with Uncertainty [6.151348127802708]
  We propose a physics-guided framework, termed Physics-guided Deep Markov Model (PgDMM) The proposed framework takes advantage of the expressive power of deep learning, while retaining the driving physics of the dynamical system.
  arXiv  Detail & Related papers  (2021-10-16T16:35:12Z)
• Physics-Integrated Variational Autoencoders for Robust and Interpretable Generative Modeling [86.9726984929758]
  We focus on the integration of incomplete physics models into deep generative models. We propose a VAE architecture in which a part of the latent space is grounded by physics. We demonstrate generative performance improvements over a set of synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2021-02-25T20:28:52Z)
• Modeling System Dynamics with Physics-Informed Neural Networks Based on Lagrangian Mechanics [3.214927790437842]
  Two main modeling approaches often fail to meet requirements: first principles methods suffer from high bias, whereas data-driven modeling tends to have high variance. We present physics-informed neural ordinary differential equations (PINODE), a hybrid model that combines the two modeling techniques to overcome the aforementioned problems. Our findings are of interest for model-based control and system identification of mechanical systems.
  arXiv  Detail & Related papers  (2020-05-29T15:10:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.