Bridging Physics-based and Data-driven modeling for Learning Dynamical Systems

• URL: http://arxiv.org/abs/2011.10616v2
• Date: Thu, 29 Apr 2021 08:09:56 GMT
• Title: Bridging Physics-based and Data-driven modeling for Learning Dynamical Systems
• Abstract summary: We formalize the learning of physics-based models as AutoODE, which leverages automatic differentiation to estimate the model parameters. Our method obtains a 57.4% reduction in mean absolute errors for 7-day ahead COVID-19 forecasting compared with the best deep learning competitor.
• Score: 36.04381259949935
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: How can we learn a dynamical system to make forecasts, when some variables are unobserved? For instance, in COVID-19, we want to forecast the number of infected and death cases but we do not know the count of susceptible and exposed people. While mechanics compartment models are widely used in epidemic modeling, data-driven models are emerging for disease forecasting. We first formalize the learning of physics-based models as AutoODE, which leverages automatic differentiation to estimate the model parameters. Through a benchmark study on COVID-19 forecasting, we notice that physics-based mechanistic models significantly outperform deep learning. Our method obtains a 57.4% reduction in mean absolute errors for 7-day ahead COVID-19 forecasting compared with the best deep learning competitor. Such performance differences highlight the generalization problem in dynamical system learning due to distribution shift. We identify two scenarios where distribution shift can occur: changes in data domain and changes in parameter domain (system dynamics). Through systematic experiments on several dynamical systems, we found that deep learning models fail to forecast well under both scenarios. While much research on distribution shift has focused on changes in the data domain, our work calls attention to rethink generalization for learning dynamical systems.

Related papers

• Interpreting learning dynamics of autoencoders: Transient scaling and emerging concepts of the Ising model [0.0]
  unsupervised autoencoders trained on microscopic spin configurations from the Ising model learn macroscopic, theory-relevant variables.<n>We quantify learning across multiple spatial scales and reveal two distinct dynamical regimes.<n>A dynamical viewpoint of learning is established in which intrinsic properties expose the effects of forced changes in representation.
  arXiv  Detail & Related papers  (2026-07-11T12:31:34Z)
• Transformers for dynamical systems learn transfer operators in-context [4.211463778317431]
  In-context learning challenges conventional understanding of learning and adaptation in physical systems.<n>We train a small two-layer, single-head transformer to forecast one dynamical system, and then evaluate its ability to forecast a different dynamical system without retraining.<n>We discover that attention-based models apply a transfer-operator forecasting strategy in-context.
  arXiv  Detail & Related papers  (2026-02-21T01:03:15Z)
• Scientific machine learning in ecological systems: A study on the predator-prey dynamics [1.4633779950109127]
  We aim to uncover the underlying differential equations without prior knowledge of the system, relying solely on training data and neural networks. We demonstrate that both Neural ODEs and UDEs can be effectively utilized for prediction and forecasting the LotkaVolterra system. We observe how UDEs outperform Neural ODEs by effectively recovering the underlying dynamics and achieving accurate forecasting with significantly less training data.
  arXiv  Detail & Related papers  (2024-11-11T10:40:45Z)
• Learning System Dynamics without Forgetting [60.08612207170659]
  Predicting trajectories of systems with unknown dynamics is crucial in various research fields, including physics and biology. We present a novel framework of Mode-switching Graph ODE (MS-GODE), which can continually learn varying dynamics. We construct a novel benchmark of biological dynamic systems, featuring diverse systems with disparate dynamics.
  arXiv  Detail & Related papers  (2024-06-30T14:55:18Z)
• Enhancing Dynamical System Modeling through Interpretable Machine Learning Augmentations: A Case Study in Cathodic Electrophoretic Deposition [0.8796261172196743]
  We introduce a comprehensive data-driven framework aimed at enhancing the modeling of physical systems. As a demonstrative application, we pursue the modeling of cathodic electrophoretic deposition (EPD), commonly known as e-coating.
  arXiv  Detail & Related papers  (2024-01-16T14:58:21Z)
• Generalizing Graph ODE for Learning Complex System Dynamics across Environments [33.63818978256567]
  GG-ODE is a machine learning framework for learning continuous multi-agent system dynamics across environments. Our model learns system dynamics using neural ordinary differential equations (ODE) parameterized by Graph Neural Networks (GNNs) Experiments over various physical simulations show that our model can accurately predict system dynamics, especially in the long range.
  arXiv  Detail & Related papers  (2023-07-10T00:29:25Z)
• Learning Latent Dynamics via Invariant Decomposition and (Spatio-)Temporal Transformers [0.6767885381740952]
  We propose a method for learning dynamical systems from high-dimensional empirical data. We focus on the setting in which data are available from multiple different instances of a system. We study behaviour through simple theoretical analyses and extensive experiments on synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2023-06-21T07:52:07Z)
• Knowledge-based Deep Learning for Modeling Chaotic Systems [7.075125892721573]
  This paper considers extreme events and their dynamics and proposes models based on deep neural networks, called knowledge-based deep learning (KDL) Our proposed KDL can learn the complex patterns governing chaotic systems by jointly training on real and simulated data. We validate our model by assessing it on three real-world benchmark datasets: El Nino sea surface temperature, San Juan Dengue viral infection, and Bjornoya daily precipitation.
  arXiv  Detail & Related papers  (2022-09-09T11:46:25Z)
• On the Generalization and Adaption Performance of Causal Models [99.64022680811281]
  Differentiable causal discovery has proposed to factorize the data generating process into a set of modules. We study the generalization and adaption performance of such modular neural causal models. Our analysis shows that the modular neural causal models outperform other models on both zero and few-shot adaptation in low data regimes.
  arXiv  Detail & Related papers  (2022-06-09T17:12:32Z)
• Capturing Actionable Dynamics with Structured Latent Ordinary Differential Equations [68.62843292346813]
  We propose a structured latent ODE model that captures system input variations within its latent representation. Building on a static variable specification, our model learns factors of variation for each input to the system, thus separating the effects of the system inputs in the latent space.
  arXiv  Detail & Related papers  (2022-02-25T20:00:56Z)
• EINNs: Epidemiologically-Informed Neural Networks [75.34199997857341]
  We introduce a new class of physics-informed neural networks-EINN-crafted for epidemic forecasting. We investigate how to leverage both the theoretical flexibility provided by mechanistic models as well as the data-driven expressability afforded by AI models.
  arXiv  Detail & Related papers  (2022-02-21T18:59:03Z)
• Learning continuous models for continuous physics [94.42705784823997]
  We develop a test based on numerical analysis theory to validate machine learning models for science and engineering applications. Our results illustrate how principled numerical analysis methods can be coupled with existing ML training/testing methodologies to validate models for science and engineering applications.
  arXiv  Detail & Related papers  (2022-02-17T07:56:46Z)
• Using Data Assimilation to Train a Hybrid Forecast System that Combines Machine-Learning and Knowledge-Based Components [52.77024349608834]
  We consider the problem of data-assisted forecasting of chaotic dynamical systems when the available data is noisy partial measurements. We show that by using partial measurements of the state of the dynamical system, we can train a machine learning model to improve predictions made by an imperfect knowledge-based model.
  arXiv  Detail & Related papers  (2021-02-15T19:56:48Z)
• An Optimal Control Approach to Learning in SIDARTHE Epidemic model [67.22168759751541]
  We propose a general approach for learning time-variant parameters of dynamic compartmental models from epidemic data. We forecast the epidemic evolution in Italy and France.
  arXiv  Detail & Related papers  (2020-10-28T10:58:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.