Unsupervised Learning of Hybrid Latent Dynamics: A Learn-to-Identify Framework

• URL: http://arxiv.org/abs/2403.08194v1
• Date: Wed, 13 Mar 2024 02:33:57 GMT
• Title: Unsupervised Learning of Hybrid Latent Dynamics: A Learn-to-Identify Framework
• Abstract summary: Modern applications increasingly require unsupervised learning of latent dynamics from high-dimensional time-series. This paper investigates the use of physics inductive bias specific to the data being modeled, and a learn-to-identify strategy that separates forecasting objectives from the data used for the identification. We combine these two strategies in a novel framework for unsupervised meta-learning of hybrid latent dynamics (Meta-HyLaD)
• Score: 9.587766468221535
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Modern applications increasingly require unsupervised learning of latent dynamics from high-dimensional time-series. This presents a significant challenge of identifiability: many abstract latent representations may reconstruct observations, yet do they guarantee an adequate identification of the governing dynamics? This paper investigates this challenge from two angles: the use of physics inductive bias specific to the data being modeled, and a learn-to-identify strategy that separates forecasting objectives from the data used for the identification. We combine these two strategies in a novel framework for unsupervised meta-learning of hybrid latent dynamics (Meta-HyLaD) with: 1) a latent dynamic function that hybridize known mathematical expressions of prior physics with neural functions describing its unknown errors, and 2) a meta-learning formulation to learn to separately identify both components of the hybrid dynamics. Through extensive experiments on five physics and one biomedical systems, we provide strong evidence for the benefits of Meta-HyLaD to integrate rich prior knowledge while identifying their gap to observed data.

Related papers

• DyMRL: Dynamic Multispace Representation Learning for Multimodal Event Forecasting in Knowledge Graph [15.450478774839324]
  We propose DyMRL, a Dynamic Multispace Representation Learning approach to efficiently acquire and fuse multimodal temporal knowledge.<n>We show that DyMRL outperforms state-of-the-art dynamic unimodal and static multimodal baseline methods.
  arXiv  Detail & Related papers  (2026-03-25T09:45:16Z)
• Uni-MMMU: A Massive Multi-discipline Multimodal Unified Benchmark [69.8473923357969]
  Unified multimodal models aim to jointly enable visual understanding and generation, yet current benchmarks rarely examine their true integration.<n>We present Uni-MMMU, a comprehensive benchmark that unfolds the bidirectional synergy between generation and understanding across eight reasoning-centric domains.
  arXiv  Detail & Related papers  (2025-10-15T17:10:35Z)
• Neural non-canonical Hamiltonian dynamics for long-time simulations [0.0]
  This work focuses on learning non-canonical Hamiltonian dynamics from data.<n>Long-term predictions require the preservation of structure both in the learned model and in numerical schemes.
  arXiv  Detail & Related papers  (2025-10-02T08:27:10Z)
• Understanding Catastrophic Interference: On the Identifibility of Latent Representations [67.05452287233122]
  Catastrophic interference, also known as catastrophic forgetting, is a fundamental challenge in machine learning.<n>We propose a novel theoretical framework that formulates catastrophic interference as an identification problem.<n>Our approach provides both theoretical guarantees and practical performance improvements across both synthetic and benchmark datasets.
  arXiv  Detail & Related papers  (2025-09-27T00:53:32Z)
• Data-driven Discovery of Digital Twins in Biomedical Research [1.1852406625172218]
  We review methodologies for automatically inferring digital twins from biological time series.<n>We evaluate algorithms according to eight biological and methodological challenges.<n>We highlight the emerging role of deep learning and large language models, which enable innovative prior knowledge integration.
  arXiv  Detail & Related papers  (2025-08-29T10:10:02Z)
• Foundation Model for Skeleton-Based Human Action Understanding [56.89025287217221]
  This paper presents a Unified Skeleton-based Dense Representation Learning framework.<n>USDRL consists of a Transformer-based Dense Spatio-Temporal (DSTE), Multi-Grained Feature Decorrelation (MG-FD), and Multi-Perspective Consistency Training (MPCT)
  arXiv  Detail & Related papers  (2025-08-18T02:42:16Z)
• Unsupervised Representation Learning of Complex Time Series for Maneuverability State Identification in Smart Mobility [0.0]
  In smart mobility, MTS plays a crucial role in providing temporal dynamics of behaviors such as maneuver patterns. In this work, we aim to address challenges associated with modeling MTS data collected from a vehicle using sensors. Our goal is to investigate the effectiveness of two distinct unsupervised representation learning approaches in identifying maneuvering states in smart mobility.
  arXiv  Detail & Related papers  (2024-08-26T15:16:18Z)
• Learning System Dynamics without Forgetting [60.08612207170659]
  Predicting trajectories of systems with unknown dynamics is crucial in various research fields, including physics and biology. We present a novel framework of Mode-switching Graph ODE (MS-GODE), which can continually learn varying dynamics. We construct a novel benchmark of biological dynamic systems, featuring diverse systems with disparate dynamics.
  arXiv  Detail & Related papers  (2024-06-30T14:55:18Z)
• Bond Graphs for multi-physics informed Neural Networks for multi-variate time series [6.775534755081169]
  Existing methods are not adapted to tasks with complex multi-physical and multi-domain phenomena. We propose a Neural Bond graph (NBgE) producing multi-physics-informed representations that can be fed into any task-specific model.
  arXiv  Detail & Related papers  (2024-05-22T12:30:25Z)
• HyPer-EP: Meta-Learning Hybrid Personalized Models for Cardiac Electrophysiology [7.230055455268642]
  We present a novel hybrid modeling framework to describe a personalized cardiac digital twin. We then present a novel meta-learning framework to enable the separate identification of both the physics-based and neural components.
  arXiv  Detail & Related papers  (2024-03-15T02:30:00Z)
• Beyond DAGs: A Latent Partial Causal Model for Multimodal Learning [80.44084021062105]
  We propose a novel latent partial causal model for multimodal data, featuring two latent coupled variables, connected by an undirected edge, to represent the transfer of knowledge across modalities.<n>Under specific statistical assumptions, we establish an identifiability result, demonstrating that representations learned by multimodal contrastive learning correspond to the latent coupled variables up to a trivial transformation.<n>Experiments on a pre-trained CLIP model embodies disentangled representations, enabling few-shot learning and improving domain generalization across diverse real-world datasets.
  arXiv  Detail & Related papers  (2024-02-09T07:18:06Z)
• Interpretable Imitation Learning with Dynamic Causal Relations [65.18456572421702]
  We propose to expose captured knowledge in the form of a directed acyclic causal graph. We also design this causal discovery process to be state-dependent, enabling it to model the dynamics in latent causal graphs. The proposed framework is composed of three parts: a dynamic causal discovery module, a causality encoding module, and a prediction module, and is trained in an end-to-end manner.
  arXiv  Detail & Related papers  (2023-09-30T20:59:42Z)
• Learning dynamics from partial observations with structured neural ODEs [5.757156314867639]
  We propose a flexible framework to incorporate a broad spectrum of physical insight into neural ODE-based system identification. We demonstrate the performance of the proposed approach on numerical simulations and on an experimental dataset from a robotic exoskeleton.
  arXiv  Detail & Related papers  (2022-05-25T07:54:10Z)
• EINNs: Epidemiologically-Informed Neural Networks [75.34199997857341]
  We introduce a new class of physics-informed neural networks-EINN-crafted for epidemic forecasting. We investigate how to leverage both the theoretical flexibility provided by mechanistic models as well as the data-driven expressability afforded by AI models.
  arXiv  Detail & Related papers  (2022-02-21T18:59:03Z)
• SyMetric: Measuring the Quality of Learnt Hamiltonian Dynamics Inferred from Vision [73.26414295633846]
  A recently proposed class of models attempts to learn latent dynamics from high-dimensional observations. Existing methods rely on image reconstruction quality, which does not always reflect the quality of the learnt latent dynamics. We develop a set of new measures, including a binary indicator of whether the underlying Hamiltonian dynamics have been faithfully captured.
  arXiv  Detail & Related papers  (2021-11-10T23:26:58Z)
• Deep Graph Memory Networks for Forgetting-Robust Knowledge Tracing [5.648636668261282]
  We propose a novel knowledge tracing model, namely emphDeep Graph Memory Network (DGMN) In this model, we incorporate a forget gating mechanism into an attention memory structure in order to capture forgetting behaviours. This model has the capability of learning relationships between latent concepts from a dynamic latent concept graph.
  arXiv  Detail & Related papers  (2021-08-18T12:04:10Z)
• TCL: Transformer-based Dynamic Graph Modelling via Contrastive Learning [87.38675639186405]
  We propose a novel graph neural network approach, called TCL, which deals with the dynamically-evolving graph in a continuous-time fashion. To the best of our knowledge, this is the first attempt to apply contrastive learning to representation learning on dynamic graphs.
  arXiv  Detail & Related papers  (2021-05-17T15:33:25Z)
• Provable Meta-Learning of Linear Representations [114.656572506859]
  We provide fast, sample-efficient algorithms to address the dual challenges of learning a common set of features from multiple, related tasks, and transferring this knowledge to new, unseen tasks. We also provide information-theoretic lower bounds on the sample complexity of learning these linear features.
  arXiv  Detail & Related papers  (2020-02-26T18:21:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.