A Unified Framework for Pattern Recovery in Penalized and Thresholded Estimation and its Geometry

• URL: http://arxiv.org/abs/2307.10158v5
• Date: Mon, 20 Oct 2025 17:42:40 GMT
• Title: A Unified Framework for Pattern Recovery in Penalized and Thresholded Estimation and its Geometry
• Abstract summary: We consider the framework of penalized estimation where the penalty term is given by a real-valued polyhedral gauge.<n>We define a novel and general notion of patterns based on subdifferentials and formalize an approach to measure pattern complexity.
• Score: 1.9949261242626628
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We consider the framework of penalized estimation where the penalty term is given by a real-valued polyhedral gauge, which encompasses methods such as LASSO, generalized LASSO, SLOPE, OSCAR, PACS and others. Each of these estimators is defined through an optimization problem and can uncover a different structure or ``pattern'' of the unknown parameter vector. We define a novel and general notion of patterns based on subdifferentials and formalize an approach to measure pattern complexity. For pattern recovery, we provide a minimal condition for a particular pattern to be detected by the procedure with positive probability, the so-called accessibility condition. Using our approach, we also introduce the stronger noiseless recovery condition. For the LASSO, it is well known that the irrepresentability condition is necessary for pattern recovery with probability larger than $1/2$ and we show that the noiseless recovery plays exactly the same role in our general framework, thereby unifying and extending the irrepresentability condition to a broad class of penalized estimators. We also show that the noiseless recovery condition can be relaxed when turning to so-called thresholded penalized estimators: we prove that the necessary condition of accessibility is already sufficient for sure pattern recovery by thresholded penalized estimation provided that the noise is small enough. Throughout the article, we demonstrate how our findings can be interpreted through a geometrical lens.

Related papers

• A general estimation framework for continuous-variable systems [32.505127447635864]
  We show that informational completeness does not guarantee statistically stable reconstruction from finite measurement data.<n>We develop a general estimation theory for continuous-variable systems in which stable reconstructibility is characterized by the POVM effects forming a measurement frame.
  arXiv  Detail & Related papers  (2026-07-21T16:58:41Z)
• Approximate Risk Minimization Over Shrinking-Thresholding Rules in Normal Mean Estimation [3.695296650216182]
  We develop an approximate risk minimization framework for shrinkage--thresholding estimation in normal mean problems.<n>We derive optimality conditions for both oracle risk and data-driven approximate risk minimization.<n>The results provide a unified risk-based framework for shrinkage, thresholding, and regularization across canonical and correlated normal mean estimation and linear regression.
  arXiv  Detail & Related papers  (2026-07-07T15:04:02Z)
• From Sparse Features to Trustworthy Proxies: Certifying SAE-Based Interpretability [53.681678236115836]
  We study when can an SAE-based explanation be treated as a faithful view of an underlying frozen LM.<n>Our framework derives an upper bound on the base model's expected risk using four measurable quantities.<n>We show that the bound becomes non-vacuous on GPT-2 Small, Gemma-2B, and Llama-3-8B at practical sample sizes.
  arXiv  Detail & Related papers  (2026-06-16T18:28:23Z)
• Next-Acceleration-Scale Prediction for Autoregressive MRI Reconstruction [52.32112533846212]
  MRI reconstruction is an inherently ill-posed inverse problem, since incomplete measurements admit many plausible solutions.<n>We address this limitation by moving reconstruction to discrete multi-scale latent space and posing it as autoregressive next-acceleration-scale prediction.<n>We show that our approach delivers improved reconstruction performance across diverse sampling patterns under extreme undersampling.
  arXiv  Detail & Related papers  (2026-05-19T04:40:50Z)
• Non-Parametric Rehearsal Learning via Conditional Mean Embeddings [88.89267783967263]
  We propose a non-parametric rehearsal learning approach for textitavoiding undesired future (AUF) problems.<n>Specifically, we use kernel machinery to reformulate the AUF objective into a unified representation that disentangles desirability modeling from action-induced distributional changes.
  arXiv  Detail & Related papers  (2026-05-09T15:30:52Z)
• Anchored Policy Optimization: Mitigating Exploration Collapse Via Support-Constrained Rectification [14.911955979675772]
  We propose Anchored Policy Optimization (APO) to shift the paradigm from global Shape Matching to Support Coverage.<n>APO breaks the accuracy-diversity trade-off, significantly improving Pass@1 while restoring the Pass@K diversity typically lost by standard policy gradient methods.
  arXiv  Detail & Related papers  (2026-02-05T14:41:57Z)
• Causal Inference as Distribution Adaptation: Optimizing ATE Risk under Propensity Uncertainty [0.0]
  We reframing ATE estimation as a textitdomain adaptation problem under distribution shift.<n>We propose the textbfJoint Robust Estimator (JRE) to train outcome models jointly.
  arXiv  Detail & Related papers  (2025-12-19T21:40:46Z)
• Convergence and Generalization of Anti-Regularization for Parametric Models [0.0]
  Anti-regularization introduces a reward term with a reversed sign into the loss function.<n>We formalize spectral safety conditions and trust-region constraints.<n>We design a lightweight safeguard that combines a projection operator with gradient clipping to guarantee stable intervention.
  arXiv  Detail & Related papers  (2025-08-24T15:34:17Z)
• Verification-Guided Falsification for Safe RL via Explainable Abstraction and Risk-Aware Exploration [8.246285288584625]
  We propose a hybrid framework that integrates explainability, model checking, and risk-guided falsification to achieve both rigor and coverage.<n>Our approach begins by constructing a human-interpretable abstraction of the RL policy using Comprehensible Abstract Policy Summarization (CAPS)<n>If no violation is detected, we cannot conclude satisfaction due to potential limitation in the abstraction and coverage of the offline dataset.
  arXiv  Detail & Related papers  (2025-06-04T00:54:01Z)
• A Probabilistic Perspective on Model Collapse [9.087950471621653]
  This paper aims to characterize the conditions under which model collapse occurs and, crucially, how it can be mitigated.<n>Under mild conditions, we rigorously show that progressively increasing the sample size at each training step is necessary to prevent model collapse.<n>We also investigate the probability that training on synthetic data yields models that outperform those trained solely on real data.
  arXiv  Detail & Related papers  (2025-05-20T05:25:29Z)
• Empirical Bayes Estimation for Lasso-Type Regularizers: Analysis of Automatic Relevance Determination [0.21485350418225244]
  This paper focuses on linear regression models with non-conjugate sparsity-inducing regularizers such as lasso and group lasso. We derive the empirical Bayes estimators for the group lasso regularized linear regression models with limited parameters.
  arXiv  Detail & Related papers  (2025-01-20T05:25:51Z)
• Rethinking State Disentanglement in Causal Reinforcement Learning [78.12976579620165]
  Causality provides rigorous theoretical support for ensuring that the underlying states can be uniquely recovered through identifiability. We revisit this research line and find that incorporating RL-specific context can reduce unnecessary assumptions in previous identifiability analyses for latent states. We propose a novel approach for general partially observable Markov Decision Processes (POMDPs) by replacing the complicated structural constraints in previous methods with two simple constraints for transition and reward preservation.
  arXiv  Detail & Related papers  (2024-08-24T06:49:13Z)
• Evaluating Model Performance Under Worst-case Subpopulations [8.615300901890253]
  We study the worst-case performance of a model over all subpopulations of a given size, defined with respect to core attributes Z. We develop a scalable yet principled two-stage estimation procedure that can evaluate the robustness of state-of-the-art models.
  arXiv  Detail & Related papers  (2024-07-01T14:24:05Z)
• Hypothesis Testing for Class-Conditional Noise Using Local Maximum Likelihood [1.8798171797988192]
  In supervised learning, automatically assessing the quality of the labels before any learning takes place remains an open research question. In this paper we show how similar procedures can be followed when the underlying model is a product of Local Maximum Likelihood Estimation. This different view allows for wider applicability of the tests by offering users access to a richer model class.
  arXiv  Detail & Related papers  (2023-12-15T22:14:58Z)
• Likelihood Ratio Confidence Sets for Sequential Decision Making [51.66638486226482]
  We revisit the likelihood-based inference principle and propose to use likelihood ratios to construct valid confidence sequences. Our method is especially suitable for problems with well-specified likelihoods. We show how to provably choose the best sequence of estimators and shed light on connections to online convex optimization.
  arXiv  Detail & Related papers  (2023-11-08T00:10:21Z)
• Conformal Prediction with Missing Values [19.18178194789968]
  We first show that the marginal coverage guarantee of conformal prediction holds on imputed data for any missingness distribution. We then show that a universally consistent quantile regression algorithm trained on the imputed data is Bayes optimal for the pinball risk.
  arXiv  Detail & Related papers  (2023-06-05T09:28:03Z)
• Toward Certified Robustness Against Real-World Distribution Shifts [65.66374339500025]
  We train a generative model to learn perturbations from data and define specifications with respect to the output of the learned model. A unique challenge arising from this setting is that existing verifiers cannot tightly approximate sigmoid activations. We propose a general meta-algorithm for handling sigmoid activations which leverages classical notions of counter-example-guided abstraction refinement.
  arXiv  Detail & Related papers  (2022-06-08T04:09:13Z)
• Mitigating multiple descents: A model-agnostic framework for risk monotonization [84.6382406922369]
  We develop a general framework for risk monotonization based on cross-validation. We propose two data-driven methodologies, namely zero- and one-step, that are akin to bagging and boosting.
  arXiv  Detail & Related papers  (2022-05-25T17:41:40Z)
• Correlation Clustering Reconstruction in Semi-Adversarial Models [70.11015369368272]
  Correlation Clustering is an important clustering problem with many applications. We study the reconstruction version of this problem in which one is seeking to reconstruct a latent clustering corrupted by random noise and adversarial modifications.
  arXiv  Detail & Related papers  (2021-08-10T14:46:17Z)
• Exact Recovery in the General Hypergraph Stochastic Block Model [92.28929858529679]
  This paper investigates fundamental limits of exact recovery in the general d-uniform hypergraph block model (d-HSBM) We show that there exists a sharp threshold such that exact recovery is achievable above the threshold and impossible below it.
  arXiv  Detail & Related papers  (2021-05-11T03:39:08Z)
• Temporal-Structure-Assisted Gradient Aggregation for Over-the-Air Federated Edge Learning [24.248673415586413]
  We introduce a Markovian probability model to characterize the intrinsic temporal structure of the model aggregation series. We develop a message passing algorithm, termed temporal-structure-assisted gradient aggregation (TSA-GA), to fulfil this estimation task. We show that the proposed TSAGA algorithm significantly outperforms the state-of-the-art, and is able to achieve comparable learning performance.
  arXiv  Detail & Related papers  (2021-03-03T09:13:27Z)
• Structural Causal Models Are (Solvable by) Credal Networks [70.45873402967297]
  Causal inferences can be obtained by standard algorithms for the updating of credal nets. This contribution should be regarded as a systematic approach to represent structural causal models by credal networks. Experiments show that approximate algorithms for credal networks can immediately be used to do causal inference in real-size problems.
  arXiv  Detail & Related papers  (2020-08-02T11:19:36Z)
• Towards a Theoretical Understanding of the Robustness of Variational Autoencoders [82.68133908421792]
  We make inroads into understanding the robustness of Variational Autoencoders (VAEs) to adversarial attacks and other input perturbations. We develop a novel criterion for robustness in probabilistic models: $r$-robustness. We show that VAEs trained using disentangling methods score well under our robustness metrics.
  arXiv  Detail & Related papers  (2020-07-14T21:22:29Z)
• Confounding-Robust Policy Evaluation in Infinite-Horizon Reinforcement Learning [70.01650994156797]
  Off- evaluation of sequential decision policies from observational data is necessary in batch reinforcement learning such as education healthcare. We develop an approach that estimates the bounds of a given policy. We prove convergence to the sharp bounds as we collect more confounded data.
  arXiv  Detail & Related papers  (2020-02-11T16:18:14Z)
• Imputation for High-Dimensional Linear Regression [8.841513006680886]
  We show that LASSO retains the minimax estimation rate in the random setting. We show that the co-root remains mphmph in this setting.
  arXiv  Detail & Related papers  (2020-01-24T19:54:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.