From STLS to Projection-based Dictionary Selection in Sparse Regression for System Identification

• URL: http://arxiv.org/abs/2512.14404v1
• Date: Tue, 16 Dec 2025 13:42:10 GMT
• Title: From STLS to Projection-based Dictionary Selection in Sparse Regression for System Identification
• Authors: Hangjun Cho, Fabio V. G. Amaral, Andrei A. Klishin, Cassio M. Oishi, Steven L. Brunton,
• Abstract summary: We revisit dictionary-based sparse regression, in particular, Sequential Threshold Least Squares (STLS)<n>We propose a score-guided library selection to provide practical guidance for data-driven modeling, with emphasis on SINDy-type algorithms.
• Score: 1.7341202786497238
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this work, we revisit dictionary-based sparse regression, in particular, Sequential Threshold Least Squares (STLS), and propose a score-guided library selection to provide practical guidance for data-driven modeling, with emphasis on SINDy-type algorithms. STLS is an algorithm to solve the $\ell_0$ sparse least-squares problem, which relies on splitting to efficiently solve the least-squares portion while handling the sparse term via proximal methods. It produces coefficient vectors whose components depend on both the projected reconstruction errors, here referred to as the scores, and the mutual coherence of dictionary terms. The first contribution of this work is a theoretical analysis of the score and dictionary-selection strategy. This could be understood in both the original and weak SINDy regime. Second, numerical experiments on ordinary and partial differential equations highlight the effectiveness of score-based screening, improving both accuracy and interpretability in dynamical system identification. These results suggest that integrating score-guided methods to refine the dictionary more accurately may help SINDy users in some cases to enhance their robustness for data-driven discovery of governing equations.

Related papers

• A Unified Probabilistic Framework for Dictionary Learning with Parsimonious Activation [10.775460285501739]
  We introduce a parsimony promoting regularizer based on the row-wise $L_infty$ norm of the coefficient matrix.<n>This additional penalty encourages entire rows of the coefficient matrix to vanish, thereby reducing the number of dictionary atoms activated across the dataset.
  arXiv  Detail & Related papers  (2025-09-30T02:46:11Z)
• Self-Boost via Optimal Retraining: An Analysis via Approximate Message Passing [58.52119063742121]
  Retraining a model using its own predictions together with the original, potentially noisy labels is a well-known strategy for improving the model performance.<n>This paper addresses the question of how to optimally combine the model's predictions and the provided labels.<n>Our main contribution is the derivation of the Bayes optimal aggregator function to combine the current model's predictions and the given labels.
  arXiv  Detail & Related papers  (2025-05-21T07:16:44Z)
• Signature Isolation Forest [4.462334751640167]
  Functional Isolation Forest (FIF) is a state-of-the-art Anomaly Detection (AD) algorithm designed for functional data.<n>We introduce textitSignature Isolation Forest, a novel AD algorithm class leveraging the rough path theory's signature transform.<n>We provide several numerical experiments, including a real-world applications benchmark showing the relevance of our methods.
  arXiv  Detail & Related papers  (2024-03-07T11:00:35Z)
• Structured Radial Basis Function Network: Modelling Diversity for Multiple Hypotheses Prediction [51.82628081279621]
  Multi-modal regression is important in forecasting nonstationary processes or with a complex mixture of distributions. A Structured Radial Basis Function Network is presented as an ensemble of multiple hypotheses predictors for regression problems. It is proved that this structured model can efficiently interpolate this tessellation and approximate the multiple hypotheses target distribution.
  arXiv  Detail & Related papers  (2023-09-02T01:27:53Z)
• Learning Unnormalized Statistical Models via Compositional Optimization [73.30514599338407]
  Noise-contrastive estimation(NCE) has been proposed by formulating the objective as the logistic loss of the real data and the artificial noise. In this paper, we study it a direct approach for optimizing the negative log-likelihood of unnormalized models.
  arXiv  Detail & Related papers  (2023-06-13T01:18:16Z)
• Dictionary-based Low-Rank Approximations and the Mixed Sparse Coding problem [7.132368785057316]
  I show how to adapt an efficient MSC solver based on the LASSO to compute Dictionary-based Matrix Factorization and Canonical Polyadic Decomposition. I show how to adapt an efficient MSC solver based on the LASSO to compute Dictionary-based Matrix Factorization and Canonical Polyadic Decomposition in the context of hyperspectral image processing and chemometrics.
  arXiv  Detail & Related papers  (2021-11-24T10:32:48Z)
• Discriminative Dictionary Learning based on Statistical Methods [0.0]
  Sparse Representation (SR) of signals or data has a well founded theory with rigorous mathematical error bounds and proofs. Training dictionaries such that they represent each class of signals with minimal loss is called Dictionary Learning (DL) MOD and K-SVD have been successfully used in reconstruction based applications in image processing like image "denoising", "inpainting"
  arXiv  Detail & Related papers  (2021-11-17T10:45:10Z)
• Dictionary Learning Using Rank-One Atomic Decomposition (ROAD) [6.367823813868024]
  Dictionary learning aims at seeking a dictionary under which the training data can be sparsely represented. Road outperforms other benchmark algorithms for both synthetic data and real data.
  arXiv  Detail & Related papers  (2021-10-25T10:29:52Z)
• Estimating leverage scores via rank revealing methods and randomization [50.591267188664666]
  We study algorithms for estimating the statistical leverage scores of rectangular dense or sparse matrices of arbitrary rank. Our approach is based on combining rank revealing methods with compositions of dense and sparse randomized dimensionality reduction transforms.
  arXiv  Detail & Related papers  (2021-05-23T19:21:55Z)
• Sparse PCA via $l_{2,p}$-Norm Regularization for Unsupervised Feature Selection [138.97647716793333]
  We propose a simple and efficient unsupervised feature selection method, by combining reconstruction error with $l_2,p$-norm regularization. We present an efficient optimization algorithm to solve the proposed unsupervised model, and analyse the convergence and computational complexity of the algorithm theoretically.
  arXiv  Detail & Related papers  (2020-12-29T04:08:38Z)
• Pairwise Supervised Hashing with Bernoulli Variational Auto-Encoder and Self-Control Gradient Estimator [62.26981903551382]
  Variational auto-encoders (VAEs) with binary latent variables provide state-of-the-art performance in terms of precision for document retrieval. We propose a pairwise loss function with discrete latent VAE to reward within-class similarity and between-class dissimilarity for supervised hashing. This new semantic hashing framework achieves superior performance compared to the state-of-the-arts.
  arXiv  Detail & Related papers  (2020-05-21T06:11:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.