PAC-Bayesian-Like Error Bound for a Class of Linear Time-Invariant Stochastic State-Space Models

• URL: http://arxiv.org/abs/2212.14838v1
• Date: Fri, 30 Dec 2022 17:37:33 GMT
• Title: PAC-Bayesian-Like Error Bound for a Class of Linear Time-Invariant Stochastic State-Space Models
• Authors: Deividas Eringis, John Leth, Zheng-Hua Tan, Rafal Wisniewski, Mihaly Petreczky
• Abstract summary: We derive a PAC-Bayesian-Like error bound for a class of dynamical systems with inputs. We discuss various consequences of this error bound.
• Score: 13.251009291060992
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: In this paper we derive a PAC-Bayesian-Like error bound for a class of stochastic dynamical systems with inputs, namely, for linear time-invariant stochastic state-space models (stochastic LTI systems for short). This class of systems is widely used in control engineering and econometrics, in particular, they represent a special case of recurrent neural networks. In this paper we 1) formalize the learning problem for stochastic LTI systems with inputs, 2) derive a PAC-Bayesian-Like error bound for such systems, 3) discuss various consequences of this error bound.

Related papers

• A finite-sample generalization bound for stable LPV systems [0.0]
  We derive a PAC bound for stable continuous-time linear parameter-varying (LPV) systems. Our bound depends on the H2 norm of the chosen class of the LPV systems, but does not depend on the time interval for which the signals are considered.
  arXiv  Detail & Related papers  (2024-05-16T12:42:36Z)
• PAC-Bayes Generalisation Bounds for Dynamical Systems Including Stable RNNs [11.338419403452239]
  We derive a PAC-Bayes bound on the generalisation gap for a special class of discrete-time non-linear dynamical systems. The proposed bound converges to zero as the dataset size increases. Unlike other available bounds the derived bound holds for non i.i.d. data (time-series) and it does not grow with the number of steps of the RNN.
  arXiv  Detail & Related papers  (2023-12-15T13:49:29Z)
• Machine learning in and out of equilibrium [58.88325379746631]
  Our study uses a Fokker-Planck approach, adapted from statistical physics, to explore these parallels. We focus in particular on the stationary state of the system in the long-time limit, which in conventional SGD is out of equilibrium. We propose a new variation of Langevin dynamics (SGLD) that harnesses without replacement minibatching.
  arXiv  Detail & Related papers  (2023-06-06T09:12:49Z)
• PAC-Bayesian bounds for learning LTI-ss systems with input from empirical loss [13.251009291060992]
  We derive a Probably Approxilmately Correct(PAC)-Bayesian error bound for linear time-invariant (LTI) dynamical systems with inputs. Such bounds are widespread in machine learning, and they are useful for characterizing the predictive power of models learned from finitely many data points.
  arXiv  Detail & Related papers  (2023-03-29T16:06:07Z)
• Formal Controller Synthesis for Markov Jump Linear Systems with Uncertain Dynamics [64.72260320446158]
  We propose a method for synthesising controllers for Markov jump linear systems. Our method is based on a finite-state abstraction that captures both the discrete (mode-jumping) and continuous (stochastic linear) behaviour of the MJLS. We apply our method to multiple realistic benchmark problems, in particular, a temperature control and an aerial vehicle delivery problem.
  arXiv  Detail & Related papers  (2022-12-01T17:36:30Z)
• Identifiability and Asymptotics in Learning Homogeneous Linear ODE Systems from Discrete Observations [114.17826109037048]
  Ordinary Differential Equations (ODEs) have recently gained a lot of attention in machine learning. theoretical aspects, e.g., identifiability and properties of statistical estimation are still obscure. This paper derives a sufficient condition for the identifiability of homogeneous linear ODE systems from a sequence of equally-spaced error-free observations sampled from a single trajectory.
  arXiv  Detail & Related papers  (2022-10-12T06:46:38Z)
• Characterizing possible failure modes in physics-informed neural networks [55.83255669840384]
  Recent work in scientific machine learning has developed so-called physics-informed neural network (PINN) models. We demonstrate that, while existing PINN methodologies can learn good models for relatively trivial problems, they can easily fail to learn relevant physical phenomena even for simple PDEs. We show that these possible failure modes are not due to the lack of expressivity in the NN architecture, but that the PINN's setup makes the loss landscape very hard to optimize.
  arXiv  Detail & Related papers  (2021-09-02T16:06:45Z)
• Consistency of mechanistic causal discovery in continuous-time using Neural ODEs [85.7910042199734]
  We consider causal discovery in continuous-time for the study of dynamical systems. We propose a causal discovery algorithm based on penalized Neural ODEs.
  arXiv  Detail & Related papers  (2021-05-06T08:48:02Z)
• PAC-Bayesian theory for stochastic LTI systems [15.01136076871549]
  We derive a PAC-Bayesian error bound for autonomous LTI state-space models. The motivation for deriving such error bounds is that they will allow deriving similar error bounds for more general dynamical systems.
  arXiv  Detail & Related papers  (2021-03-23T21:59:21Z)
• LSTM for Model-Based Anomaly Detection in Cyber-Physical Systems [4.020523898765404]
  Anomaly detection is the task of detecting data which differs from the normal behaviour of a system in a given context. Long Short-Term Memory (LSTM) neural networks have been shown to be particularly useful to learn time sequences. We analyse the approach on artificial and real data.
  arXiv  Detail & Related papers  (2020-10-29T15:26:08Z)
• Time-Reversal Symmetric ODE Network [138.02741983098454]
  Time-reversal symmetry is a fundamental property that frequently holds in classical and quantum mechanics. We propose a novel loss function that measures how well our ordinary differential equation (ODE) networks comply with this time-reversal symmetry. We show that, even for systems that do not possess the full time-reversal symmetry, TRS-ODENs can achieve better predictive performances over baselines.
  arXiv  Detail & Related papers  (2020-07-22T12:19:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.