PAC-Bayesian Bounds for Learning Partially Observed Stochastic Linear Time-Invariant State-Space Systems with Inputs and Sub-Gaussian Noise

• URL: http://arxiv.org/abs/2609.08740v1
• Date: Tue, 08 Sep 2026 13:32:20 GMT
• Title: PAC-Bayesian Bounds for Learning Partially Observed Stochastic Linear Time-Invariant State-Space Systems with Inputs and Sub-Gaussian Noise
• Abstract summary: We derive a PAC-Bayesian error bound for partially observed linear time-invariant (LTI) dynamical systems in state-space form with inputs and sub-Gaussian noise.<n>Such bounds are widespread in machine learning, and they are useful for characterizing the predictive power of models learned from finitely many data points.
• Score: 1.551190544492585
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper we derive a Probably Approximately Correct (PAC)-Bayesian error bound for partially observed linear time-invariant (LTI) stochastic dynamical systems in state-space form with inputs and sub-Gaussian noise. Such bounds are widespread in machine learning, and they are useful for characterizing the predictive power of models learned from finitely many data points. The bound derived in this paper relates the expectation of prediction errors with the prediction error generated by the model on the data used for learning. In addition, we show that it can also be used to derive bounds for the parameter estimation error. In turn, this allows us to provide finite-sample error bounds for the prediction error and parameter estimation error for a wide class of system identification algorithms. Furthermore, as LTI systems are a sub-class of recurrent neural networks (RNNs), these error bounds could be a first step towards PAC-Bayesian bounds for RNNs.

Related papers

• Tensor network characterization and mitigation of readout errors [16.364682124857485]
  Readout errors are a major bottleneck to extracting reliable information from near-term quantum processors.<n>We present a unified tensor-network framework that models the readout process as a matrix product operator (MPO)<n>Experiments on a superconducting processor and numerical simulations up to 20 qubits show that the MPO model captures correlated readout errors that uncorrelated models miss.
  arXiv  Detail & Related papers  (2026-06-24T15:45:47Z)
• Generative Modeling with Continuous Flows: Sample Complexity of Flow Matching [60.37045080890305]
  We provide the first analysis of the sample complexity for flow-matching based generative models.<n>We decompose the velocity field estimation error into neural-network approximation error, statistical error due to the finite sample size, and optimization error due to the finite number of optimization steps for estimating the velocity field.
  arXiv  Detail & Related papers  (2025-12-01T05:14:25Z)
• Error Adjustment Based on Spatiotemporal Correlation Fusion for Traffic Forecasting [22.37553946699755]
  A general assumption of training the said forecasting models via mean error estimation is that the errors across time steps and spatial positions are unrelated.<n>This paper proposes Stemporally Autorelated Error Adjustment (SAEA), a novel and general framework designed to systematically autocorrelated prediction errors in traffic forecasting.
  arXiv  Detail & Related papers  (2025-10-25T23:48:50Z)
• MultiPDENet: PDE-embedded Learning with Multi-time-stepping for Accelerated Flow Simulation [48.41289705783405]
  We propose a PDE-embedded network with multiscale time stepping (MultiPDENet)<n>In particular, we design a convolutional filter based on the structure of finite difference with a small number of parameters to optimize.<n>A Physics Block with a 4th-order Runge-Kutta integrator at the fine time scale is established that embeds the structure of PDEs to guide the prediction.
  arXiv  Detail & Related papers  (2025-01-27T12:15:51Z)
• Bayesian Model Parameter Learning in Linear Inverse Problems: Application in EEG Focal Source Imaging [49.1574468325115]
  Inverse problems can be described as limited-data problems in which the signal of interest cannot be observed directly.<n>We studied a linear inverse problem that included an unknown non-linear model parameter.<n>We utilized a Bayesian model-based learning approach that allowed signal recovery and subsequently estimation of the model parameter.
  arXiv  Detail & Related papers  (2025-01-07T18:14:24Z)
• Dreaming Learning [41.94295877935867]
  Introducing new information to a machine learning system can interfere with previously stored data.<n>We propose a training algorithm inspired by Stuart Kauffman's notion of the Adjacent Possible.<n>It predisposes the neural network to smoothly accept and integrate data sequences with different statistical characteristics than expected.
  arXiv  Detail & Related papers  (2024-10-23T09:17:31Z)
• A PAC-Bayesian Perspective on the Interpolating Information Criterion [54.548058449535155]
  We show how a PAC-Bayes bound is obtained for a general class of models, characterizing factors which influence performance in the interpolating regime. We quantify how the test error for overparameterized models achieving effectively zero training error depends on the quality of the implicit regularization imposed by e.g. the combination of model, parameter-initialization scheme.
  arXiv  Detail & Related papers  (2023-11-13T01:48:08Z)
• Kalman Filter for Online Classification of Non-Stationary Data [101.26838049872651]
  In Online Continual Learning (OCL) a learning system receives a stream of data and sequentially performs prediction and training steps. We introduce a probabilistic Bayesian online learning model by using a neural representation and a state space model over the linear predictor weights. In experiments in multi-class classification we demonstrate the predictive ability of the model and its flexibility to capture non-stationarity.
  arXiv  Detail & Related papers  (2023-06-14T11:41:42Z)
• PAC-Bayesian bounds for learning LTI-ss systems with input from empirical loss [13.251009291060992]
  We derive a Probably Approxilmately Correct(PAC)-Bayesian error bound for linear time-invariant (LTI) dynamical systems with inputs. Such bounds are widespread in machine learning, and they are useful for characterizing the predictive power of models learned from finitely many data points.
  arXiv  Detail & Related papers  (2023-03-29T16:06:07Z)
• Uncovering the Missing Pattern: Unified Framework Towards Trajectory Imputation and Prediction [60.60223171143206]
  Trajectory prediction is a crucial undertaking in understanding entity movement or human behavior from observed sequences. Current methods often assume that the observed sequences are complete while ignoring the potential for missing values. This paper presents a unified framework, the Graph-based Conditional Variational Recurrent Neural Network (GC-VRNN), which can perform trajectory imputation and prediction simultaneously.
  arXiv  Detail & Related papers  (2023-03-28T14:27:27Z)
• PAC-Bayesian-Like Error Bound for a Class of Linear Time-Invariant Stochastic State-Space Models [13.251009291060992]
  We derive a PAC-Bayesian-Like error bound for a class of dynamical systems with inputs. We discuss various consequences of this error bound.
  arXiv  Detail & Related papers  (2022-12-30T17:37:33Z)
• Imputation-Free Learning from Incomplete Observations [73.15386629370111]
  We introduce the importance of guided gradient descent (IGSGD) method to train inference from inputs containing missing values without imputation. We employ reinforcement learning (RL) to adjust the gradients used to train the models via back-propagation. Our imputation-free predictions outperform the traditional two-step imputation-based predictions using state-of-the-art imputation methods.
  arXiv  Detail & Related papers  (2021-07-05T12:44:39Z)
• PAC-Bayesian theory for stochastic LTI systems [15.01136076871549]
  We derive a PAC-Bayesian error bound for autonomous LTI state-space models. The motivation for deriving such error bounds is that they will allow deriving similar error bounds for more general dynamical systems.
  arXiv  Detail & Related papers  (2021-03-23T21:59:21Z)
• Improving predictions of Bayesian neural nets via local linearization [79.21517734364093]
  We argue that the Gauss-Newton approximation should be understood as a local linearization of the underlying Bayesian neural network (BNN) Because we use this linearized model for posterior inference, we should also predict using this modified model instead of the original one. We refer to this modified predictive as "GLM predictive" and show that it effectively resolves common underfitting problems of the Laplace approximation.
  arXiv  Detail & Related papers  (2020-08-19T12:35:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.