Sampling-Free Probabilistic Deep State-Space Models

• URL: http://arxiv.org/abs/2309.08256v1
• Date: Fri, 15 Sep 2023 09:06:23 GMT
• Title: Sampling-Free Probabilistic Deep State-Space Models
• Authors: Andreas Look, Melih Kandemir, Barbara Rakitsch, Jan Peters
• Abstract summary: A Probabilistic Deep SSM generalizes to dynamical systems of unknown parametric form. We propose the first deterministic inference algorithm for models of this type.
• Score: 28.221200872943825
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Many real-world dynamical systems can be described as State-Space Models (SSMs). In this formulation, each observation is emitted by a latent state, which follows first-order Markovian dynamics. A Probabilistic Deep SSM (ProDSSM) generalizes this framework to dynamical systems of unknown parametric form, where the transition and emission models are described by neural networks with uncertain weights. In this work, we propose the first deterministic inference algorithm for models of this type. Our framework allows efficient approximations for training and testing. We demonstrate in our experiments that our new method can be employed for a variety of tasks and enjoys a superior balance between predictive performance and computational budget.

Related papers

• Latent Space Energy-based Neural ODEs [73.01344439786524]
  This paper introduces a novel family of deep dynamical models designed to represent continuous-time sequence data. We train the model using maximum likelihood estimation with Markov chain Monte Carlo. Experiments on oscillating systems, videos and real-world state sequences (MuJoCo) illustrate that ODEs with the learnable energy-based prior outperform existing counterparts.
  arXiv  Detail & Related papers  (2024-09-05T18:14:22Z)
• Enhanced Prediction of Multi-Agent Trajectories via Control Inference and State-Space Dynamics [14.694200929205975]
  This paper introduces a novel methodology for trajectory forecasting based on state-space dynamic system modeling. To enhance the precision of state estimations within the dynamic system, the paper also presents a novel modeling technique for control variables. The proposed approach ingeniously integrates graph neural networks with state-space models, effectively capturing the complexities of multi-agent interactions.
  arXiv  Detail & Related papers  (2024-08-08T08:33:02Z)
• HiPPO-Prophecy: State-Space Models can Provably Learn Dynamical Systems in Context [0.5416466085090772]
  This work explores the in-context learning capabilities of State Space Models (SSMs) We introduce a novel weight construction for SSMs, enabling them to predict the next state of any dynamical system. We extend the HiPPO framework to demonstrate that continuous SSMs can approximate the derivative of any input signal.
  arXiv  Detail & Related papers  (2024-07-12T15:56:11Z)
• Understanding Self-attention Mechanism via Dynamical System Perspective [58.024376086269015]
  Self-attention mechanism (SAM) is widely used in various fields of artificial intelligence. We show that intrinsic stiffness phenomenon (SP) in the high-precision solution of ordinary differential equations (ODEs) also widely exists in high-performance neural networks (NN) We show that the SAM is also a stiffness-aware step size adaptor that can enhance the model's representational ability to measure intrinsic SP.
  arXiv  Detail & Related papers  (2023-08-19T08:17:41Z)
• Learning minimal representations of stochastic processes with variational autoencoders [52.99137594502433]
  We introduce an unsupervised machine learning approach to determine the minimal set of parameters required to describe a process. Our approach enables for the autonomous discovery of unknown parameters describing processes.
  arXiv  Detail & Related papers  (2023-07-21T14:25:06Z)
• Cheap and Deterministic Inference for Deep State-Space Models of Interacting Dynamical Systems [38.23826389188657]
  We present a deep state-space model which employs graph neural networks in order to model the underlying interacting dynamical system. The predictive distribution is multimodal and has the form of a Gaussian mixture model, where the moments of the Gaussian components can be computed via deterministic moment matching rules. Our moment matching scheme can be exploited for sample-free inference, leading to more efficient and stable training compared to Monte Carlo alternatives.
  arXiv  Detail & Related papers  (2023-05-02T20:30:23Z)
• Capturing dynamical correlations using implicit neural representations [85.66456606776552]
  We develop an artificial intelligence framework which combines a neural network trained to mimic simulated data from a model Hamiltonian with automatic differentiation to recover unknown parameters from experimental data. In doing so, we illustrate the ability to build and train a differentiable model only once, which then can be applied in real-time to multi-dimensional scattering data.
  arXiv  Detail & Related papers  (2023-04-08T07:55:36Z)
• Distributed Bayesian Learning of Dynamic States [65.7870637855531]
  The proposed algorithm is a distributed Bayesian filtering task for finite-state hidden Markov models. It can be used for sequential state estimation, as well as for modeling opinion formation over social networks under dynamic environments.
  arXiv  Detail & Related papers  (2022-12-05T19:40:17Z)
• Learning Stochastic Dynamics with Statistics-Informed Neural Network [0.4297070083645049]
  We introduce a machine-learning framework named statistics-informed neural network (SINN) for learning dynamics from data. We devise mechanisms for training the neural network model to reproduce the correct emphstatistical behavior of a target process. We show that the obtained reduced-order model can be trained on temporally coarse-grained data and hence is well suited for rare-event simulations.
  arXiv  Detail & Related papers  (2022-02-24T18:21:01Z)
• Likelihood-Free Inference in State-Space Models with Unknown Dynamics [71.94716503075645]
  We introduce a method for inferring and predicting latent states in state-space models where observations can only be simulated, and transition dynamics are unknown. We propose a way of doing likelihood-free inference (LFI) of states and state prediction with a limited number of simulations.
  arXiv  Detail & Related papers  (2021-11-02T12:33:42Z)
• Deep Probabilistic Time Series Forecasting using Augmented Recurrent Input for Dynamic Systems [12.319812075685956]
  We combine the advances in both deep generative models and state space model (SSM) to come up with a novel, data-driven deep probabilistic sequence model. Specially, we follow the popular encoder-decoder generative structure to build the recurrent neural networks (RNN) assisted variational sequence model. In order to alleviate the issue of inconsistency between training and predicting, we (i) propose using a hybrid output as input at next time step, which brings training and predicting into alignment.
  arXiv  Detail & Related papers  (2021-06-03T23:41:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.