Tensor Decompositions Meet Control Theory: Learning General Mixtures of Linear Dynamical Systems

• URL: http://arxiv.org/abs/2307.06538v2
• Date: Sun, 23 Jul 2023 05:35:08 GMT
• Title: Tensor Decompositions Meet Control Theory: Learning General Mixtures of Linear Dynamical Systems
• Authors: Ainesh Bakshi, Allen Liu, Ankur Moitra, Morris Yau
• Abstract summary: We give a new approach to learning mixtures of linear dynamical systems based on tensor decompositions. Our algorithm succeeds without strong separation conditions on the components, and can be used to compete with the Bayes optimal clustering of the trajectories.
• Score: 19.47235707806519
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently Chen and Poor initiated the study of learning mixtures of linear dynamical systems. While linear dynamical systems already have wide-ranging applications in modeling time-series data, using mixture models can lead to a better fit or even a richer understanding of underlying subpopulations represented in the data. In this work we give a new approach to learning mixtures of linear dynamical systems that is based on tensor decompositions. As a result, our algorithm succeeds without strong separation conditions on the components, and can be used to compete with the Bayes optimal clustering of the trajectories. Moreover our algorithm works in the challenging partially-observed setting. Our starting point is the simple but powerful observation that the classic Ho-Kalman algorithm is a close relative of modern tensor decomposition methods for learning latent variable models. This gives us a playbook for how to extend it to work with more complicated generative models.

Related papers

• Dynamic Post-Hoc Neural Ensemblers [55.15643209328513]
  In this study, we explore employing neural networks as ensemble methods. Motivated by the risk of learning low-diversity ensembles, we propose regularizing the model by randomly dropping base model predictions. We demonstrate this approach lower bounds the diversity within the ensemble, reducing overfitting and improving generalization capabilities.
  arXiv  Detail & Related papers  (2024-10-06T15:25:39Z)
• Class-Incremental Mixture of Gaussians for Deep Continual Learning [15.49323098362628]
  We propose end-to-end incorporation of the mixture of Gaussians model into the continual learning framework. We show that our model can effectively learn in memory-free scenarios with fixed extractors.
  arXiv  Detail & Related papers  (2023-07-09T04:33:19Z)
• Cheap and Deterministic Inference for Deep State-Space Models of Interacting Dynamical Systems [38.23826389188657]
  We present a deep state-space model which employs graph neural networks in order to model the underlying interacting dynamical system. The predictive distribution is multimodal and has the form of a Gaussian mixture model, where the moments of the Gaussian components can be computed via deterministic moment matching rules. Our moment matching scheme can be exploited for sample-free inference, leading to more efficient and stable training compared to Monte Carlo alternatives.
  arXiv  Detail & Related papers  (2023-05-02T20:30:23Z)
• Dynamic Bayesian Learning and Calibration of Spatiotemporal Mechanistic System [0.0]
  We develop an approach for fully learning and calibration of mechanistic models based on noisy observations. We demonstrate this flexibility through solving problems arising in the analysis of ordinary and partial nonlinear differential equations.
  arXiv  Detail & Related papers  (2022-08-12T23:17:46Z)
• Learning Sparse Nonlinear Dynamics via Mixed-Integer Optimization [3.7565501074323224]
  We propose an exact formulation of the SINDyDy problem using mixed-integer optimization (MIO) to solve the sparsity constrained regression problem to provable optimality in seconds. We illustrate the dramatic improvement of our approach in accurate model discovery while being more sample efficient, robust to noise, and flexible in accommodating physical constraints.
  arXiv  Detail & Related papers  (2022-06-01T01:43:45Z)
• Supervised DKRC with Images for Offline System Identification [77.34726150561087]
  Modern dynamical systems are becoming increasingly non-linear and complex. There is a need for a framework to model these systems in a compact and comprehensive representation for prediction and control. Our approach learns these basis functions using a supervised learning approach.
  arXiv  Detail & Related papers  (2021-09-06T04:39:06Z)
• Gone Fishing: Neural Active Learning with Fisher Embeddings [55.08537975896764]
  There is an increasing need for active learning algorithms that are compatible with deep neural networks. This article introduces BAIT, a practical representation of tractable, and high-performing active learning algorithm for neural networks.
  arXiv  Detail & Related papers  (2021-06-17T17:26:31Z)
• Learning Unstable Dynamics with One Minute of Data: A Differentiation-based Gaussian Process Approach [47.045588297201434]
  We show how to exploit the differentiability of Gaussian processes to create a state-dependent linearized approximation of the true continuous dynamics. We validate our approach by iteratively learning the system dynamics of an unstable system such as a 9-D segway.
  arXiv  Detail & Related papers  (2021-03-08T05:08:47Z)
• Using Data Assimilation to Train a Hybrid Forecast System that Combines Machine-Learning and Knowledge-Based Components [52.77024349608834]
  We consider the problem of data-assisted forecasting of chaotic dynamical systems when the available data is noisy partial measurements. We show that by using partial measurements of the state of the dynamical system, we can train a machine learning model to improve predictions made by an imperfect knowledge-based model.
  arXiv  Detail & Related papers  (2021-02-15T19:56:48Z)
• Learning Stable Deep Dynamics Models [91.90131512825504]
  We propose an approach for learning dynamical systems that are guaranteed to be stable over the entire state space. We show that such learning systems are able to model simple dynamical systems and can be combined with additional deep generative models to learn complex dynamics.
  arXiv  Detail & Related papers  (2020-01-17T00:04:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.