PAC-Bayesian bounds for learning LTI-ss systems with input from empirical loss

• URL: http://arxiv.org/abs/2303.16816v1
• Date: Wed, 29 Mar 2023 16:06:07 GMT
• Title: PAC-Bayesian bounds for learning LTI-ss systems with input from empirical loss
• Authors: Deividas Eringis, John Leth, Zheng-Hua Tan, Rafael Wisniewski, Mihaly Petreczky
• Abstract summary: We derive a Probably Approxilmately Correct(PAC)-Bayesian error bound for linear time-invariant (LTI) dynamical systems with inputs. Such bounds are widespread in machine learning, and they are useful for characterizing the predictive power of models learned from finitely many data points.
• Score: 13.251009291060992
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: In this paper we derive a Probably Approxilmately Correct(PAC)-Bayesian error bound for linear time-invariant (LTI) stochastic dynamical systems with inputs. Such bounds are widespread in machine learning, and they are useful for characterizing the predictive power of models learned from finitely many data points. In particular, with the bound derived in this paper relates future average prediction errors with the prediction error generated by the model on the data used for learning. In turn, this allows us to provide finite-sample error bounds for a wide class of learning/system identification algorithms. Furthermore, as LTI systems are a sub-class of recurrent neural networks (RNNs), these error bounds could be a first step towards PAC-Bayesian bounds for RNNs.

Related papers

• Correlation-aware Spatial-Temporal Graph Learning for Multivariate Time-series Anomaly Detection [67.60791405198063]
  We propose a correlation-aware spatial-temporal graph learning (termed CST-GL) for time series anomaly detection. CST-GL explicitly captures the pairwise correlations via a multivariate time series correlation learning module. A novel anomaly scoring component is further integrated into CST-GL to estimate the degree of an anomaly in a purely unsupervised manner.
  arXiv  Detail & Related papers  (2023-07-17T11:04:27Z)
• PAC-Bayesian-Like Error Bound for a Class of Linear Time-Invariant Stochastic State-Space Models [13.251009291060992]
  We derive a PAC-Bayesian-Like error bound for a class of dynamical systems with inputs. We discuss various consequences of this error bound.
  arXiv  Detail & Related papers  (2022-12-30T17:37:33Z)
• Efficient learning of nonlinear prediction models with time-series privileged information [11.679648862014655]
  We show that for prediction in linear-Gaussian dynamical systems, a LuPI learner with access to intermediate time series data is never worse than any unbiased classical learner. We propose algorithms based on random features and representation learning for the case when this map is unknown.
  arXiv  Detail & Related papers  (2022-09-15T05:56:36Z)
• Certified machine learning: A posteriori error estimation for physics-informed neural networks [0.0]
  PINNs are known to be robust for smaller training sets, derive better generalization problems, and are faster to train. We show that using PINNs in comparison with purely data-driven neural networks is not only favorable for training performance but allows us to extract significant information on the quality of the approximated solution.
  arXiv  Detail & Related papers  (2022-03-31T14:23:04Z)
• Adaptive Anomaly Detection for Internet of Things in Hierarchical Edge Computing: A Contextual-Bandit Approach [81.5261621619557]
  We propose an adaptive anomaly detection scheme with hierarchical edge computing (HEC) We first construct multiple anomaly detection DNN models with increasing complexity, and associate each of them to a corresponding HEC layer. Then, we design an adaptive model selection scheme that is formulated as a contextual-bandit problem and solved by using a reinforcement learning policy network.
  arXiv  Detail & Related papers  (2021-08-09T08:45:47Z)
• Imputation-Free Learning from Incomplete Observations [73.15386629370111]
  We introduce the importance of guided gradient descent (IGSGD) method to train inference from inputs containing missing values without imputation. We employ reinforcement learning (RL) to adjust the gradients used to train the models via back-propagation. Our imputation-free predictions outperform the traditional two-step imputation-based predictions using state-of-the-art imputation methods.
  arXiv  Detail & Related papers  (2021-07-05T12:44:39Z)
• Gone Fishing: Neural Active Learning with Fisher Embeddings [55.08537975896764]
  There is an increasing need for active learning algorithms that are compatible with deep neural networks. This article introduces BAIT, a practical representation of tractable, and high-performing active learning algorithm for neural networks.
  arXiv  Detail & Related papers  (2021-06-17T17:26:31Z)
• PAC-Bayesian theory for stochastic LTI systems [15.01136076871549]
  We derive a PAC-Bayesian error bound for autonomous LTI state-space models. The motivation for deriving such error bounds is that they will allow deriving similar error bounds for more general dynamical systems.
  arXiv  Detail & Related papers  (2021-03-23T21:59:21Z)
• Learning Reasoning Strategies in End-to-End Differentiable Proving [50.9791149533921]
  Conditional Theorem Provers learn optimal rule selection strategy via gradient-based optimisation. We show that Conditional Theorem Provers are scalable and yield state-of-the-art results on the CLUTRR dataset.
  arXiv  Detail & Related papers  (2020-07-13T16:22:14Z)
• High-level Modeling of Manufacturing Faults in Deep Neural Network Accelerators [2.6258269516366557]
  Google's Unit Processing (TPU) is a neural network accelerator that uses systolic array-based matrix multiplication hardware for computation in its crux. Manufacturing faults at any state element of the matrix multiplication unit can cause unexpected errors in these inference networks. We propose a formal model of permanent faults and their propagation in a TPU using the Discrete-Time Markov Chain (DTMC) formalism.
  arXiv  Detail & Related papers  (2020-06-05T18:11:14Z)
• Belief Propagation Reloaded: Learning BP-Layers for Labeling Problems [83.98774574197613]
  We take one of the simplest inference methods, a truncated max-product Belief propagation, and add what is necessary to make it a proper component of a deep learning model. This BP-Layer can be used as the final or an intermediate block in convolutional neural networks (CNNs) The model is applicable to a range of dense prediction problems, is well-trainable and provides parameter-efficient and robust solutions in stereo, optical flow and semantic segmentation.
  arXiv  Detail & Related papers  (2020-03-13T13:11:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.