An L-BFGS-B approach for linear and nonlinear system identification under $\ell_1$- and group-Lasso regularization

• URL: http://arxiv.org/abs/2403.03827v2
• Date: Wed, 17 Jul 2024 05:42:14 GMT
• Title: An L-BFGS-B approach for linear and nonlinear system identification under $\ell_1$- and group-Lasso regularization
• Authors: Alberto Bemporad,
• Abstract summary: We propose a very efficient numerical method for identifying linear and nonlinear discrete-time state-space models. A Python implementation of the proposed identification method is available in the package jax-sysid.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we propose a very efficient numerical method based on the L-BFGS-B algorithm for identifying linear and nonlinear discrete-time state-space models, possibly under $\ell_1$- and group-Lasso regularization for reducing model complexity. For the identification of linear models, we show that, compared to classical linear subspace methods, the approach often provides better results, is much more general in terms of the loss and regularization terms used (such as penalties for enforcing system stability), and is also more stable from a numerical point of view. The proposed method not only enriches the existing set of linear system identification tools but can also be applied to identifying a very broad class of parametric nonlinear state-space models, including recurrent neural networks. We illustrate the approach on synthetic and experimental datasets and apply it to solve a challenging industrial robot benchmark for nonlinear multi-input/multi-output system identification. A Python implementation of the proposed identification method is available in the package jax-sysid, available at https://github.com/bemporad/jax-sysid.

Related papers

• Learning Controlled Stochastic Differential Equations [61.82896036131116]
  This work proposes a novel method for estimating both drift and diffusion coefficients of continuous, multidimensional, nonlinear controlled differential equations with non-uniform diffusion. We provide strong theoretical guarantees, including finite-sample bounds for (L2), (Linfty), and risk metrics, with learning rates adaptive to coefficients' regularity. Our method is available as an open-source Python library.
  arXiv  Detail & Related papers  (2024-11-04T11:09:58Z)
• Exact identification of nonlinear dynamical systems by Trimmed Lasso [0.0]
  Identification of nonlinear dynamical systems has been popularized by sparse identification of the nonlinear dynamics (SINDy) algorithm. E-SINDy was proposed for model identification, handling finite, highly noisy data. In this paper, we demonstrate that the Trimmed Lasso for robust identification of models (TRIM) can provide exact recovery under more severe noise, finite data, and multicollinearity as opposed to E-SINDy.
  arXiv  Detail & Related papers  (2023-08-03T17:37:18Z)
• An Accelerated Doubly Stochastic Gradient Method with Faster Explicit Model Identification [97.28167655721766]
  We propose a novel doubly accelerated gradient descent (ADSGD) method for sparsity regularized loss minimization problems. We first prove that ADSGD can achieve a linear convergence rate and lower overall computational complexity.
  arXiv  Detail & Related papers  (2022-08-11T22:27:22Z)
• A Priori Denoising Strategies for Sparse Identification of Nonlinear Dynamical Systems: A Comparative Study [68.8204255655161]
  We investigate and compare the performance of several local and global smoothing techniques to a priori denoise the state measurements. We show that, in general, global methods, which use the entire measurement data set, outperform local methods, which employ a neighboring data subset around a local point.
  arXiv  Detail & Related papers  (2022-01-29T23:31:25Z)
• Training Recurrent Neural Networks by Sequential Least Squares and the Alternating Direction Method of Multipliers [0.20305676256390928]
  We propose the use of convex and twice-differentiable loss and regularization terms for determining optimal hidden network parameters. We combine sequential least squares with alternating direction multipliers. The performance of the algorithm is tested in a nonlinear system identification benchmark.
  arXiv  Detail & Related papers  (2021-12-31T08:43:04Z)
• Sparse Bayesian Deep Learning for Dynamic System Identification [14.040914364617418]
  This paper proposes a sparse Bayesian treatment of deep neural networks (DNNs) for system identification. The proposed Bayesian approach offers a principled way to alleviate the challenges by marginal likelihood/model evidence approximation. The effectiveness of the proposed Bayesian approach is demonstrated on several linear and nonlinear systems identification benchmarks.
  arXiv  Detail & Related papers  (2021-07-27T16:09:48Z)
• Sample-Efficient Reinforcement Learning Is Feasible for Linearly Realizable MDPs with Limited Revisiting [60.98700344526674]
  Low-complexity models such as linear function representation play a pivotal role in enabling sample-efficient reinforcement learning. In this paper, we investigate a new sampling protocol, which draws samples in an online/exploratory fashion but allows one to backtrack and revisit previous states in a controlled and infrequent manner. We develop an algorithm tailored to this setting, achieving a sample complexity that scales practicallyly with the feature dimension, the horizon, and the inverse sub-optimality gap, but not the size of the state/action space.
  arXiv  Detail & Related papers  (2021-05-17T17:22:07Z)
• Identification of Probability weighted ARX models with arbitrary domains [75.91002178647165]
  PieceWise Affine models guarantees universal approximation, local linearity and equivalence to other classes of hybrid system. In this work, we focus on the identification of PieceWise Auto Regressive with eXogenous input models with arbitrary regions (NPWARX) The architecture is conceived following the Mixture of Expert concept, developed within the machine learning field.
  arXiv  Detail & Related papers  (2020-09-29T12:50:33Z)
• Sparse Methods for Automatic Relevance Determination [0.0]
  We first review automatic relevance determination (ARD) and analytically demonstrate the need to additional regularization or thresholding to achieve sparse models. We then discuss two classes of methods, regularization based and thresholding based, which build on ARD to learn parsimonious solutions to linear problems.
  arXiv  Detail & Related papers  (2020-05-18T14:08:49Z)
• Nonlinear system identification with regularized Tensor Network B-splines [2.817412580574242]
  The TNBS-NARX model is validated through the identification of the cascaded watertank benchmark nonlinear system. It achieves state-of-the-art performance while identifying a 16-dimensional B-spline surface in 4 seconds on a standard desktop computer.
  arXiv  Detail & Related papers  (2020-03-17T09:22:20Z)
• Multiscale Non-stationary Stochastic Bandits [83.48992319018147]
  We propose a novel multiscale changepoint detection method for the non-stationary linear bandit problems, called Multiscale-LinUCB. Experimental results show that our proposed Multiscale-LinUCB algorithm outperforms other state-of-the-art algorithms in non-stationary contextual environments.
  arXiv  Detail & Related papers  (2020-02-13T00:24:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.