Out-of-Domain Generalization in Dynamical Systems Reconstruction

• URL: http://arxiv.org/abs/2402.18377v2
• Date: Fri, 7 Jun 2024 23:38:25 GMT
• Title: Out-of-Domain Generalization in Dynamical Systems Reconstruction
• Authors: Niclas Göring, Florian Hess, Manuel Brenner, Zahra Monfared, Daniel Durstewitz,
• Abstract summary: We provide a formal framework that addresses generalization in DSR. We show that black-box DL techniques, without adequate structural priors, generally will not be able to learn a generalizing DSR model.
• Score: 8.397468572544614
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In science we are interested in finding the governing equations, the dynamical rules, underlying empirical phenomena. While traditionally scientific models are derived through cycles of human insight and experimentation, recently deep learning (DL) techniques have been advanced to reconstruct dynamical systems (DS) directly from time series data. State-of-the-art dynamical systems reconstruction (DSR) methods show promise in capturing invariant and long-term properties of observed DS, but their ability to generalize to unobserved domains remains an open challenge. Yet, this is a crucial property we would expect from any viable scientific theory. In this work, we provide a formal framework that addresses generalization in DSR. We explain why and how out-of-domain (OOD) generalization (OODG) in DSR profoundly differs from OODG considered elsewhere in machine learning. We introduce mathematical notions based on topological concepts and ergodic theory to formalize the idea of learnability of a DSR model. We formally prove that black-box DL techniques, without adequate structural priors, generally will not be able to learn a generalizing DSR model. We also show this empirically, considering major classes of DSR algorithms proposed so far, and illustrate where and why they fail to generalize across the whole phase space. Our study provides the first comprehensive mathematical treatment of OODG in DSR, and gives a deeper conceptual understanding of where the fundamental problems in OODG lie and how they could possibly be addressed in practice.

Related papers

• Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMs [69.10441885629787]
  Retrieval-Augmented Generation (RAG) lifts the factuality of Large Language Models (LLMs) by injecting external knowledge.<n>It falls short on problems that demand multi-step inference; conversely, purely reasoning-oriented approaches often hallucinate or mis-ground facts.<n>This survey synthesizes both strands under a unified reasoning-retrieval perspective.
  arXiv  Detail & Related papers  (2025-07-13T03:29:41Z)
• DrSR: LLM based Scientific Equation Discovery with Dual Reasoning from Data and Experience [14.093206703519103]
  DrSR is a framework that combines data-driven insight with reflective learning to enhance both robustness and discovery capability.<n> Experiments across interdisciplinary datasets in physics, chemistry, biology, and materials science demonstrate that DrSR substantially improves the valid equation rate.
  arXiv  Detail & Related papers  (2025-06-04T04:52:34Z)
• Weight-Space Linear Recurrent Neural Networks [0.5937476291232799]
  WARP (Weight-space Adaptive Recurrent Prediction) is a powerful framework that unifies weight-space learning with linear recurrence.<n>We show that WARP matches or surpasses state-of-the-art baselines on diverse classification tasks.
  arXiv  Detail & Related papers  (2025-06-01T20:13:28Z)
• Conservation-informed Graph Learning for Spatiotemporal Dynamics Prediction [84.26340606752763]
  In this paper, we introduce the conservation-informed GNN (CiGNN), an end-to-end explainable learning framework. The network is designed to conform to the general symmetry conservation law via symmetry where conservative and non-conservative information passes over a multiscale space by a latent temporal marching strategy. Results demonstrate that CiGNN exhibits remarkable baseline accuracy and generalizability, and is readily applicable to learning for prediction of varioustemporal dynamics.
  arXiv  Detail & Related papers  (2024-12-30T13:55:59Z)
• A Survey on All-in-One Image Restoration: Taxonomy, Evaluation and Future Trends [67.43992456058541]
  Image restoration (IR) refers to the process of improving visual quality of images while removing degradation, such as noise, blur, weather effects, and so on. Traditional IR methods typically target specific types of degradation, which limits their effectiveness in real-world scenarios with complex distortions. The all-in-one image restoration (AiOIR) paradigm has emerged, offering a unified framework that adeptly addresses multiple degradation types.
  arXiv  Detail & Related papers  (2024-10-19T11:11:09Z)
• Towards a theory of learning dynamics in deep state space models [12.262490032020832]
  State space models (SSMs) have shown remarkable empirical performance on many long sequence modeling tasks. This work is a step toward a theory of learning dynamics in deep state space models.
  arXiv  Detail & Related papers  (2024-07-10T00:01:56Z)
• Learning Discrete Concepts in Latent Hierarchical Models [73.01229236386148]
  Learning concepts from natural high-dimensional data holds potential in building human-aligned and interpretable machine learning models. We formalize concepts as discrete latent causal variables that are related via a hierarchical causal model. We substantiate our theoretical claims with synthetic data experiments.
  arXiv  Detail & Related papers  (2024-06-01T18:01:03Z)
• Understanding the differences in Foundation Models: Attention, State Space Models, and Recurrent Neural Networks [50.29356570858905]
  We introduce the Dynamical Systems Framework (DSF), which allows a principled investigation of all these architectures in a common representation. We provide principled comparisons between softmax attention and other model classes, discussing the theoretical conditions under which softmax attention can be approximated. This shows the DSF's potential to guide the systematic development of future more efficient and scalable foundation models.
  arXiv  Detail & Related papers  (2024-05-24T17:19:57Z)
• Learning the Topology and Behavior of Discrete Dynamical Systems [30.424671907681688]
  We focus on the problem of learning the behavior and the underlying topology of a black-box system. We show that, in general, this learning problem is computationally intractable. Our results provide a theoretical foundation for learning both the behavior and topology of discrete dynamical systems.
  arXiv  Detail & Related papers  (2024-02-18T19:31:26Z)
• Discovering Interpretable Physical Models using Symbolic Regression and Discrete Exterior Calculus [55.2480439325792]
  We propose a framework that combines Symbolic Regression (SR) and Discrete Exterior Calculus (DEC) for the automated discovery of physical models. DEC provides building blocks for the discrete analogue of field theories, which are beyond the state-of-the-art applications of SR to physical problems. We prove the effectiveness of our methodology by re-discovering three models of Continuum Physics from synthetic experimental data.
  arXiv  Detail & Related papers  (2023-10-10T13:23:05Z)
• Learning Neural Constitutive Laws From Motion Observations for Generalizable PDE Dynamics [97.38308257547186]
  Many NN approaches learn an end-to-end model that implicitly models both the governing PDE and material models. We argue that the governing PDEs are often well-known and should be explicitly enforced rather than learned. We introduce a new framework termed "Neural Constitutive Laws" (NCLaw) which utilizes a network architecture that strictly guarantees standard priors.
  arXiv  Detail & Related papers  (2023-04-27T17:42:24Z)
• Towards Out-Of-Distribution Generalization: A Survey [46.329995334444156]
  Out-of-Distribution generalization is an emerging topic of machine learning research. This paper represents the first comprehensive, systematic review of OOD generalization.
  arXiv  Detail & Related papers  (2021-08-31T05:28:42Z)
• Generalized Real-World Super-Resolution through Adversarial Robustness [107.02188934602802]
  We present Robust Super-Resolution, a method that leverages the generalization capability of adversarial attacks to tackle real-world SR. Our novel framework poses a paradigm shift in the development of real-world SR methods. By using a single robust model, we outperform state-of-the-art specialized methods on real-world benchmarks.
  arXiv  Detail & Related papers  (2021-08-25T22:43:20Z)
• Physics-Guided Deep Learning for Dynamical Systems: A survey [5.733401663293044]
  Traditional physics-based models are interpretable but rely on rigid assumptions. Deep learning provides novel alternatives for efficiently recognizing complex patterns and emulating nonlinear dynamics. It aims to take the best from both physics-based modeling and state-of-the-art DL models to better solve scientific problems.
  arXiv  Detail & Related papers  (2021-07-02T20:59:03Z)
• Modern Koopman Theory for Dynamical Systems [2.5889588665122725]
  We provide an overview of modern Koopman operator theory, describing recent theoretical and algorithmic developments. We also discuss key advances and challenges in the rapidly growing field of machine learning.
  arXiv  Detail & Related papers  (2021-02-24T06:18:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.