Adaptive Iterative Hard Thresholding for Online High-dimensional Quantile Regression

• URL: http://arxiv.org/abs/2606.28652v1
• Date: Fri, 26 Jun 2026 23:44:10 GMT
• Title: Adaptive Iterative Hard Thresholding for Online High-dimensional Quantile Regression
• Abstract summary: We propose an online sparse-regression framework that alternates subgradient updates with adaptively scheduled hard-thresholding steps.<n>We develop the theory for high-dimensional online quantile regression, a challenging setting in which the loss is nonsmooth.
• Score: 1.0067421338825542
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Online high-dimensional regression requires algorithms that can update sequentially while preserving structural sparsity. We propose \textit{Adaptive Iterative Hard Thresholding (AIHT)}, an online sparse-regression framework that alternates stochastic subgradient updates with adaptively scheduled hard-thresholding steps. The key idea is to separate support discovery from local refinement: early in the learning process, AIHT delays thresholding so that weak but informative coordinates have time to accumulate signal, while later it increases the projection frequency to stabilize the sparse estimator and exploit local curvature. We develop the theory for high-dimensional online quantile regression, a challenging setting in which the loss is nonsmooth and the data may exhibit heterogeneity or heavy-tailed noise. Under restricted curvature and gradient-leakage conditions, AIHT remains in an inflated sparse cone, exhibits a two-phase convergence behavior, and attains logarithmic regret for the sliding-window objective. Simulations for online quantile regression, together with threshold-scheduling ablations, support the proposed mechanism and illustrate its advantage over standard online sparse-learning baselines.

Related papers

• Across the Loss Landscape with Progressive Growth [51.366966420881646]
  We show how grow-and-optimize strategies to train toward flatter regions by viewing growth as progressive relaxation.<n>We empirically validate these predictions in controlled toy basins and in a realistic ResNet/CI-100 setting.<n>Although progressive subspace growth reliably produces flatter solutions, curvature reductions do not universally translate into improved test performance, highlighting subtleties in the flatness-generalization connection.
  arXiv  Detail & Related papers  (2026-08-25T13:50:48Z)
• Online Learning with Gradient-Variation Interval Regret [54.59204826681113]
  We propose the first online learning algorithm that achieves an interval regret bound scaling with gradient variation.<n>Our method employs a simple and efficient two-layer online ensemble structure that achieves strong theoretical guarantees.
  arXiv  Detail & Related papers  (2026-06-02T16:16:45Z)
• Clipping Bottleneck: Stabilizing RLVR via Stochastic Recovery of Near-Boundary Signals [83.0127582612634]
  Near-boundary Rescue (NSR) is a minimal, plug-and-play modification that retains out-of-bound tokens to recover lost signals.<n>NSR substantially improves training stability and delivers consistent gains over strong baselines such as DAPO and GSPO.
  arXiv  Detail & Related papers  (2026-05-21T16:45:31Z)
• JAWS: Enhancing Long-term Rollout of Neural Operators via Spatially-Adaptive Jacobian Regularization [0.0]
  Jacobian-Adaptive Weighting for Stability (JAWS) is a probabilistic regularization strategy designed to mitigate these limitations.<n>By framing operator learning as Maximum A Posteriori (MAP) estimation with spatially heteroscedastic uncertainty, JAWS dynamically modulates the regularization strength based on local physical complexity.<n> Experiments demonstrate that this spatially-adaptive prior serves as an effective spectral pre-conditioner, which reduces the base operator's burden of handling high-frequency instabilities.
  arXiv  Detail & Related papers  (2026-03-04T06:15:09Z)
• When Learning Hurts: Fixed-Pole RNN for Real-Time Online Training [58.25341036646294]
  We analytically examine why learning recurrent poles does not provide tangible benefits in data and empirically offer real-time learning scenarios.<n>We show that fixed-pole networks achieve superior performance with lower training complexity, making them more suitable for online real-time tasks.
  arXiv  Detail & Related papers  (2026-02-25T00:15:13Z)
• Forecasting as Rendering: A 2D Gaussian Splatting Framework for Time Series Forecasting [79.37674445572462]
  Time series forecasting (TSF) remains a challenging problem due to the intricate entanglement of intraperiod-fluctuations and interperiod-trends.<n>Treating reshaped tensors as static images results in a topological mismatch.<n> relying on uniform fixed-size representations allocates modeling capacity inefficiently.<n>TimeGS is a novel framework that fundamentally shifts the forecasting paradigm from regression to 2D generative rendering.
  arXiv  Detail & Related papers  (2026-02-10T14:13:36Z)
• LatentTrack: Sequential Weight Generation via Latent Filtering [0.0]
  LatentTrack (LT) is a sequential neural architecture for online probabilistic prediction under nonstationary dynamics.<n> LT performs causal Bayesian filtering in a low-dimensional latent space and uses a lightweight hypernetwork to generate predictive model parameters at each time step.<n> LT consistently calibrated lower negative log-likelihood and mean squared error than stateful sequential and static uncertainty-aware baselines.
  arXiv  Detail & Related papers  (2026-01-31T02:22:59Z)
• Adaptive Benign Overfitting (ABO): Overparameterized RLS for Online Learning in Non-stationary Time-series [0.0]
  ABO is highly accurate (comparable to baseline kernel methods) while achieving speed improvements of between 20 and 40 percent.<n>Results provide a unified view linking adaptive filtering, kernel approximation, and benign overfitting within a stable online learning framework.
  arXiv  Detail & Related papers  (2026-01-29T15:58:01Z)
• Trainability-Oriented Hybrid Quantum Regression via Geometric Preconditioning and Curriculum Optimization [0.6445605125467574]
  Quantum neural networks (QNNs) have attracted growing interest for scientific machine learning.<n>In regression settings they often suffer from limited trainability under noisy gradients and ill-conditioned optimization.<n>We propose a hybrid quantum-classical regression framework designed to mitigate these bottlenecks.
  arXiv  Detail & Related papers  (2026-01-17T07:32:18Z)
• Gradient-Variation Online Learning under Generalized Smoothness [56.38427425920781]
  gradient-variation online learning aims to achieve regret guarantees that scale with variations in gradients of online functions. Recent efforts in neural network optimization suggest a generalized smoothness condition, allowing smoothness to correlate with gradient norms. We provide the applications for fast-rate convergence in games and extended adversarial optimization.
  arXiv  Detail & Related papers  (2024-08-17T02:22:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.