Hidden Markov Nonlinear ICA: Unsupervised Learning from Nonstationary Time Series

• URL: http://arxiv.org/abs/2006.12107v1
• Date: Mon, 22 Jun 2020 10:01:15 GMT
• Title: Hidden Markov Nonlinear ICA: Unsupervised Learning from Nonstationary Time Series
• Authors: Hermanni H\"alv\"a and Aapo Hyv\"arinen
• Abstract summary: We show how to combine nonlinear Independent Component Analysis with a Hidden Markov Model. We prove identifiability of the proposed model for a general mixing nonlinearity, such as a neural network. We achieve a new nonlinear ICA framework which is unsupervised, more efficient, as well as able to model underlying temporal dynamics.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances in nonlinear Independent Component Analysis (ICA) provide a principled framework for unsupervised feature learning and disentanglement. The central idea in such works is that the latent components are assumed to be independent conditional on some observed auxiliary variables, such as the time-segment index. This requires manual segmentation of data into non-stationary segments which is computationally expensive, inaccurate and often impossible. These models are thus not fully unsupervised. We remedy these limitations by combining nonlinear ICA with a Hidden Markov Model, resulting in a model where a latent state acts in place of the observed segment-index. We prove identifiability of the proposed model for a general mixing nonlinearity, such as a neural network. We also show how maximum likelihood estimation of the model can be done using the expectation-maximization algorithm. Thus, we achieve a new nonlinear ICA framework which is unsupervised, more efficient, as well as able to model underlying temporal dynamics.

Related papers

• Data-driven Nonlinear Model Reduction using Koopman Theory: Integrated Control Form and NMPC Case Study [56.283944756315066]
  We propose generic model structures combining delay-coordinate encoding of measurements and full-state decoding to integrate reduced Koopman modeling and state estimation. A case study demonstrates that our approach provides accurate control models and enables real-time capable nonlinear model predictive control of a high-purity cryogenic distillation column.
  arXiv  Detail & Related papers  (2024-01-09T11:54:54Z)
• Learning Linear Causal Representations from Interventions under General Nonlinear Mixing [52.66151568785088]
  We prove strong identifiability results given unknown single-node interventions without access to the intervention targets. This is the first instance of causal identifiability from non-paired interventions for deep neural network embeddings.
  arXiv  Detail & Related papers  (2023-06-04T02:32:12Z)
• Capturing dynamical correlations using implicit neural representations [85.66456606776552]
  We develop an artificial intelligence framework which combines a neural network trained to mimic simulated data from a model Hamiltonian with automatic differentiation to recover unknown parameters from experimental data. In doing so, we illustrate the ability to build and train a differentiable model only once, which then can be applied in real-time to multi-dimensional scattering data.
  arXiv  Detail & Related papers  (2023-04-08T07:55:36Z)
• Interpretable reduced-order modeling with time-scale separation [9.889399863931676]
  Partial Differential Equations (PDEs) with high dimensionality are commonly encountered in computational physics and engineering. We propose a data-driven scheme that automates the identification of the time-scales involved. We show that this data-driven scheme can automatically learn the independent processes that decompose a system of linear ODEs.
  arXiv  Detail & Related papers  (2023-03-03T19:23:59Z)
• Neural Abstractions [72.42530499990028]
  We present a novel method for the safety verification of nonlinear dynamical models that uses neural networks to represent abstractions of their dynamics. We demonstrate that our approach performs comparably to the mature tool Flow* on existing benchmark nonlinear models.
  arXiv  Detail & Related papers  (2023-01-27T12:38:09Z)
• Adaptive deep learning for nonlinear time series models [0.0]
  We develop a theory for adaptive nonparametric estimation of the mean function of a non-stationary and nonlinear time series model using deep neural networks (DNNs) We derive minimax lower bounds for estimating mean functions belonging to a wide class of nonlinear autoregressive (AR) models.
  arXiv  Detail & Related papers  (2022-07-06T09:58:58Z)
• Nonlinear proper orthogonal decomposition for convection-dominated flows [0.0]
  We propose an end-to-end Galerkin-free model combining autoencoders with long short-term memory networks for dynamics. Our approach not only improves the accuracy, but also significantly reduces the computational cost of training and testing.
  arXiv  Detail & Related papers  (2021-10-15T18:05:34Z)
• Disentangling Identifiable Features from Noisy Data with Structured Nonlinear ICA [4.340954888479091]
  We introduce a new general identifiable framework for principled disentanglement referred to as Structured Independent Component Analysis (SNICA) Our contribution is to extend the identifiability theory of deep generative models for a very broad class of structured models. We establish the major result that identifiability for this framework holds even in the presence of noise of unknown distribution.
  arXiv  Detail & Related papers  (2021-06-17T15:56:57Z)
• I Don't Need $\mathbf{u}$: Identifiable Non-Linear ICA Without Side Information [13.936583337756883]
  We introduce a new approach for identifiable non-linear ICA models. In particular, we focus on generative models which perform clustering in their latent space.
  arXiv  Detail & Related papers  (2021-06-09T17:22:08Z)
• Hessian Eigenspectra of More Realistic Nonlinear Models [73.31363313577941]
  We make a emphprecise characterization of the Hessian eigenspectra for a broad family of nonlinear models. Our analysis takes a step forward to identify the origin of many striking features observed in more complex machine learning models.
  arXiv  Detail & Related papers  (2021-03-02T06:59:52Z)
• Anomaly Detection of Time Series with Smoothness-Inducing Sequential Variational Auto-Encoder [59.69303945834122]
  We present a Smoothness-Inducing Sequential Variational Auto-Encoder (SISVAE) model for robust estimation and anomaly detection of time series. Our model parameterizes mean and variance for each time-stamp with flexible neural networks. We show the effectiveness of our model on both synthetic datasets and public real-world benchmarks.
  arXiv  Detail & Related papers  (2021-02-02T06:15:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.