Renewable Lasso without Batch-Number Constraints: A Gradient-Enhanced Approach

• URL: http://arxiv.org/abs/2606.11738v1
• Date: Wed, 10 Jun 2026 07:15:55 GMT
• Title: Renewable Lasso without Batch-Number Constraints: A Gradient-Enhanced Approach
• Abstract summary: We study online estimation for high-dimensional generalized linear models with streaming data.<n>For the non-distributed setting, we propose a surrogate loss that approximates the cumulative loss using only historical summaries.<n>We then extend the method to distributed streaming data under the master-client architecture.
• Score: 18.206702877104643
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We study online estimation for high-dimensional generalized linear models with streaming data. First, for the non-distributed setting, we propose a gradient-enhanced surrogate loss that approximates the cumulative loss using only historical summaries, which modifies and improves upon the existing renewable estimation approach for the same model in the high-dimensional setting, and removes the batch-number constraint in previous studies. We then extend the method to distributed streaming data under the master-client architecture, where batches are partitioned across sites and only summaries (gradient vectors) are exchanged. Instead of directing applying the popular method of Jordan et al. (2019) to the surrogate quadratic loss, our adjusted approach does not require the clients to compute the full surrogate loss. We derive non-asymptotic error bounds under the high-dimensional scaling, without the stringent constraint on the number of batches in the previous studies. Simulation results under linear and logistic models, together with a real-data application, show improved accuracy over existing renewable estimators.

Related papers

• Joint Model and Data Sparsification via the Marginal Likelihood [53.29070892356214]
  We propose jointly learning individual feature and sample relevancies, enabling simultaneous model and data sparsification.<n>This symmetric pruning of model and data offers a natural extension that preserves conjugacy.<n> Empirical results across diverse regression tasks affirm that a joint ARD approach consistently yields both sparse and robust prediction models.
  arXiv  Detail & Related papers  (2026-05-28T13:26:53Z)
• Efficient Group Lasso Regularized Rank Regression with Data-Driven Parameter Determination [2.847099287022546]
  High-dimensional regression often suffers from heavy-tailed noise and outliers, which can severely undermine the reliability of least-squares based methods.<n>To improve robustness, we adopt a non-smooth Wilcoxon score based rank objective and incorporate structured group sparsity regularization.<n>We also introduce a data-driven, simulation-based tuning rule and further establish a finite-sample error bound for the resulting estimator.
  arXiv  Detail & Related papers  (2025-10-13T15:45:58Z)
• Total Robustness in Bayesian Nonlinear Regression for Measurement Error Problems under Model Misspecification [7.233732121762458]
  We present the first Bayesian nonparametric framework targeting total robustness that tackles all three challenges in general nonlinear regression.<n>A gradient-based algorithm enables efficient computations; simulations and two real-world studies show lower estimation error and reduced estimation sensitivity to misspecification.<n>The framework, therefore, offers a practical and interpretable paradigm for trustworthy regression when data and models are jointly imperfect.
  arXiv  Detail & Related papers  (2025-10-03T15:58:40Z)
• Heterogeneous Self-Supervised Acoustic Pre-Training with Local Constraints [64.15709757611369]
  We propose a new self-supervised pre-training approach to dealing with heterogeneous data.<n>The proposed approach can significantly improve the adaptivity of the self-supervised pre-trained model for the downstream supervised fine-tuning tasks.
  arXiv  Detail & Related papers  (2025-08-27T15:48:50Z)
• Optimal lower bounds for logistic log-likelihoods [1.3124513975412255]
  The logit transform is arguably the most widely-employed link function beyond linear settings. It is still unclear whether tangent lower bounds sharper than quadratic ones can be derived without undermining the tractability of the resulting minorizer. This article addresses such a challenging question through the design and study of a novel piece-wise quadratic lower bound.
  arXiv  Detail & Related papers  (2024-10-14T09:09:33Z)
• Online Tensor Inference [0.0]
  Traditional offline learning, involving the storage and utilization of all data in each computational iteration, becomes impractical for high-dimensional tensor data. Existing low-rank tensor methods lack the capability for statistical inference in an online fashion. Our approach employs Gradient Descent (SGD) to enable efficient real-time data processing without extensive memory requirements.
  arXiv  Detail & Related papers  (2023-12-28T16:37:48Z)
• Consensus-Adaptive RANSAC [104.87576373187426]
  We propose a new RANSAC framework that learns to explore the parameter space by considering the residuals seen so far via a novel attention layer. The attention mechanism operates on a batch of point-to-model residuals, and updates a per-point estimation state to take into account the consensus found through a lightweight one-step transformer.
  arXiv  Detail & Related papers  (2023-07-26T08:25:46Z)
• Refining Amortized Posterior Approximations using Gradient-Based Summary Statistics [0.9176056742068814]
  We present an iterative framework to improve the amortized approximations of posterior distributions in the context of inverse problems. We validate our method in a controlled setting by applying it to a stylized problem, and observe improved posterior approximations with each iteration.
  arXiv  Detail & Related papers  (2023-05-15T15:47:19Z)
• Deep Equilibrium Optical Flow Estimation [80.80992684796566]
  Recent state-of-the-art (SOTA) optical flow models use finite-step recurrent update operations to emulate traditional algorithms. These RNNs impose large computation and memory overheads, and are not directly trained to model such stable estimation. We propose deep equilibrium (DEQ) flow estimators, an approach that directly solves for the flow as the infinite-level fixed point of an implicit layer.
  arXiv  Detail & Related papers  (2022-04-18T17:53:44Z)
• Improving Generalization via Uncertainty Driven Perturbations [107.45752065285821]
  We consider uncertainty-driven perturbations of the training data points. Unlike loss-driven perturbations, uncertainty-guided perturbations do not cross the decision boundary. We show that UDP is guaranteed to achieve the robustness margin decision on linear models.
  arXiv  Detail & Related papers  (2022-02-11T16:22:08Z)
• Scalable Marginal Likelihood Estimation for Model Selection in Deep Learning [78.83598532168256]
  Marginal-likelihood based model-selection is rarely used in deep learning due to estimation difficulties. Our work shows that marginal likelihoods can improve generalization and be useful when validation data is unavailable.
  arXiv  Detail & Related papers  (2021-04-11T09:50:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.