Dealing with Collinearity in Large-Scale Linear System Identification Using Gaussian Regression

• URL: http://arxiv.org/abs/2302.10959v1
• Date: Tue, 21 Feb 2023 19:35:47 GMT
• Title: Dealing with Collinearity in Large-Scale Linear System Identification Using Gaussian Regression
• Authors: Wenqi Cao, Gianluigi Pillonetto
• Abstract summary: We consider estimation of networks consisting of several interconnected dynamic systems. We develop a strategy cast in a Bayesian regularization framework where any impulse response is seen as realization of a zero-mean Gaussian process. We design a novel Markov chain Monte Carlo scheme able to reconstruct the impulse responses posterior by efficiently dealing with collinearity.
• Score: 3.04585143845864
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Many problems arising in control and cybernetics require the determination of a mathematical model of the application. This has often to be performed starting from input-output data, leading to a task known as system identification in the engineering literature. One emerging topic in this field is estimation of networks consisting of several interconnected dynamic systems. We consider the linear setting assuming that system outputs are the result of many correlated inputs, hence making system identification severely ill-conditioned. This is a scenario often encountered when modeling complex cybernetics systems composed by many sub-units with feedback and algebraic loops. We develop a strategy cast in a Bayesian regularization framework where any impulse response is seen as realization of a zero-mean Gaussian process. Any covariance is defined by the so called stable spline kernel which includes information on smooth exponential decay. We design a novel Markov chain Monte Carlo scheme able to reconstruct the impulse responses posterior by efficiently dealing with collinearity. Our scheme relies on a variation of the Gibbs sampling technique: beyond considering blocks forming a partition of the parameter space, some other (overlapping) blocks are also updated on the basis of the level of collinearity of the system inputs. Theoretical properties of the algorithm are studied obtaining its convergence rate. Numerical experiments are included using systems containing hundreds of impulse responses and highly correlated inputs.

Related papers

• Learning Controlled Stochastic Differential Equations [61.82896036131116]
  This work proposes a novel method for estimating both drift and diffusion coefficients of continuous, multidimensional, nonlinear controlled differential equations with non-uniform diffusion. We provide strong theoretical guarantees, including finite-sample bounds for (L2), (Linfty), and risk metrics, with learning rates adaptive to coefficients' regularity. Our method is available as an open-source Python library.
  arXiv  Detail & Related papers  (2024-11-04T11:09:58Z)
• Weak Collocation Regression for Inferring Stochastic Dynamics with L\'{e}vy Noise [8.15076267771005]
  We propose a weak form of the Fokker-Planck (FP) equation for extracting dynamics with L'evy noise. Our approach can simultaneously distinguish mixed noise types, even in multi-dimensional problems.
  arXiv  Detail & Related papers  (2024-03-13T06:54:38Z)
• Decoherence time control by interconnection for finite-level quantum memory systems [0.7252027234425334]
  This paper is concerned with open quantum systems whose dynamic variables have an algebraic structure. The Hamiltonian and the operators of coupling the system to the external bosonic fields depend linearly on the system variables. We consider the decoherence time over the energy parameters of the system and obtain a condition under which the zero Hamiltonian provides a suboptimal solution.
  arXiv  Detail & Related papers  (2023-11-04T01:21:55Z)
• Iterative Sketching for Secure Coded Regression [66.53950020718021]
  We propose methods for speeding up distributed linear regression. Specifically, we randomly rotate the basis of the system of equations and then subsample blocks, to simultaneously secure the information and reduce the dimension of the regression problem.
  arXiv  Detail & Related papers  (2023-08-08T11:10:42Z)
• Formal Controller Synthesis for Markov Jump Linear Systems with Uncertain Dynamics [64.72260320446158]
  We propose a method for synthesising controllers for Markov jump linear systems. Our method is based on a finite-state abstraction that captures both the discrete (mode-jumping) and continuous (stochastic linear) behaviour of the MJLS. We apply our method to multiple realistic benchmark problems, in particular, a temperature control and an aerial vehicle delivery problem.
  arXiv  Detail & Related papers  (2022-12-01T17:36:30Z)
• Distributed Nonlinear State Estimation in Electric Power Systems using Graph Neural Networks [1.1470070927586016]
  This paper introduces an original graph neural network based SE implementation over the augmented factor graph of the nonlinear power system SE. The proposed regression model has linear computational complexity during the inference time once trained, with a possibility of distributed implementation.
  arXiv  Detail & Related papers  (2022-07-23T08:54:24Z)
• Dealing with collinearity in large-scale linear system identification using Bayesian regularization [3.04585143845864]
  We consider the identification of large-scale linear and stable dynamic systems whose outputs may be the result of many correlated inputs. We develop a strategy based on Bayesian regularization where any impulse response is modeled as the realization of a zero-mean Gaussian process. We then design a new Markov chain Monte Carlo scheme that deals with collinearity and is able to efficiently reconstruct the posterior of the impulse responses.
  arXiv  Detail & Related papers  (2022-03-25T13:11:26Z)
• Capturing Actionable Dynamics with Structured Latent Ordinary Differential Equations [68.62843292346813]
  We propose a structured latent ODE model that captures system input variations within its latent representation. Building on a static variable specification, our model learns factors of variation for each input to the system, thus separating the effects of the system inputs in the latent space.
  arXiv  Detail & Related papers  (2022-02-25T20:00:56Z)
• System identification using Bayesian neural networks with nonparametric noise models [0.0]
  We propose a nonparametric approach for system identification in discrete time nonlinear random dynamical systems. A Gibbs sampler for posterior inference is proposed and its effectiveness is illustrated in simulated and real time series.
  arXiv  Detail & Related papers  (2021-04-25T09:49:50Z)
• A Novel Anomaly Detection Algorithm for Hybrid Production Systems based on Deep Learning and Timed Automata [73.38551379469533]
  DAD:DeepAnomalyDetection is a new approach for automatic model learning and anomaly detection in hybrid production systems. It combines deep learning and timed automata for creating behavioral model from observations. The algorithm has been applied to few data sets including two from real systems and has shown promising results.
  arXiv  Detail & Related papers  (2020-10-29T08:27:43Z)
• Multipole Graph Neural Operator for Parametric Partial Differential Equations [57.90284928158383]
  One of the main challenges in using deep learning-based methods for simulating physical systems is formulating physics-based data. We propose a novel multi-level graph neural network framework that captures interaction at all ranges with only linear complexity. Experiments confirm our multi-graph network learns discretization-invariant solution operators to PDEs and can be evaluated in linear time.
  arXiv  Detail & Related papers  (2020-06-16T21:56:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.