SPPCSO: Adaptive Penalized Estimation Method for High-Dimensional Correlated Data

• URL: http://arxiv.org/abs/2603.06251v1
• Date: Fri, 06 Mar 2026 13:10:05 GMT
• Title: SPPCSO: Adaptive Penalized Estimation Method for High-Dimensional Correlated Data
• Abstract summary: This work proposes the Single-Parametric Principal Component Selection Operator (SPPCSO), an innovative penalized estimation method.<n>It integrates single-parametric principal component regression and $L_1$ regularization to adaptively adjust the shrinkage factor by incorporating principal component information.<n>It achieves a balance between variable selection and coefficient estimation, ensuring model stability and robust estimation even in high-dimensional, high-noise environments.
• Score: 7.338284747998038
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the rise of high-dimensional correlated data, multicollinearity poses a significant challenge to model stability, often leading to unstable estimation and reduced predictive accuracy. This work proposes the Single-Parametric Principal Component Selection Operator (SPPCSO), an innovative penalized estimation method that integrates single-parametric principal component regression and $L_{1}$ regularization to adaptively adjust the shrinkage factor by incorporating principal component information. This approach achieves a balance between variable selection and coefficient estimation, ensuring model stability and robust estimation even in high-dimensional, high-noise environments. The primary contribution lies in addressing the instability of traditional variable selection methods when applied to high-noise, high-dimensional correlated data. Theoretically, our method exhibits selection consistency and achieves a smaller estimation error bound compared to traditional penalized estimation approaches. Extensive numerical experiments demonstrate that SPPCSO not only delivers stable and reliable estimation in high-noise settings but also accurately distinguishes signal variables from noise variables in group-effect structured data with highly correlated noise variables, effectively eliminating redundant variables and achieving more stable variable selection. Furthermore, SPPCSO successfully identifies disease-associated genes in gene expression data analysis, showcasing strong practical value. The results indicate that SPPCSO serves as an ideal tool for high-dimensional variable selection, offering an efficient and interpretable solution for modeling correlated data.

Related papers

• Joint Model and Data Sparsification via the Marginal Likelihood [53.29070892356214]
  We propose jointly learning individual feature and sample relevancies, enabling simultaneous model and data sparsification.<n>This symmetric pruning of model and data offers a natural extension that preserves conjugacy.<n> Empirical results across diverse regression tasks affirm that a joint ARD approach consistently yields both sparse and robust prediction models.
  arXiv  Detail & Related papers  (2026-05-28T13:26:53Z)
• Hierarchical Probabilistic Principal Component Analysis of Longitudinal Data [13.477352592943616]
  In longitudinal studies, a large number of variables are measured repeatedly over time, with substantial missing data.<n>We introduce hierarchical probabilistic principal component analysis (HPPCA), a two-level probabilistic factor model that explicitly separates between-subject variance from time-varying within-subject dynamics.<n> Simulation studies demonstrated that HPPCA robustly recovers model parameters subspaces and substantially outperforms both standard PPCA and multivariate functional PCA in imputation accuracy, even under heavy missingness and model misspecification.
  arXiv  Detail & Related papers  (2026-04-23T19:08:12Z)
• An Interpretable and Stable Framework for Sparse Principal Component Analysis [7.338284747998038]
  Sparse principal component analysis (SPCA) addresses the poor interpretability and variable redundancy in high-dimensional data.<n>We propose SP-SPCA, a method that introduces a single equilibrium parameter into the regularization framework to adaptively adjust variable penalties.
  arXiv  Detail & Related papers  (2026-03-14T07:27:05Z)
• A Stable Neural Statistical Dependence Estimator for Autoencoder Feature Analysis [6.697324698653934]
  dependence measures like mutual information are ideal for analyzing autoencoders, but they can be ill-posed for deterministic, static, noise-free networks.<n>We adopt the variational (Gaussian) formulation that makes dependence among inputs, latents, and reconstructions measurable.<n>We propose a stable neural dependence estimator based on an orthonormal density-ratio decomposition.
  arXiv  Detail & Related papers  (2026-03-12T01:37:19Z)
• Towards a data-scale independent regulariser for robust sparse identification of non-linear dynamics [0.0]
  Data normalisation can severely distort the discovery of governing equations by magnitudebased sparse regression methods.<n>We introduce the Sequential Thresholding of Coefficient of Variation (STCV), a novel, computationally efficient sparse regression algorithm.<n>We show that STCV-based methods can successfully identify the correct, sparse physical laws even when other methods fail.
  arXiv  Detail & Related papers  (2026-03-05T14:11:22Z)
• Not All Preferences Are Created Equal: Stability-Aware and Gradient-Efficient Alignment for Reasoning Models [52.48582333951919]
  We propose a dynamic framework designed to enhance alignment reliability by maximizing the Signal-to-Noise Ratio of policy updates.<n>SAGE (Stability-Aware Gradient Efficiency) integrates a coarse-grained curriculum mechanism that refreshes candidate pools based on model competence.<n> Experiments on multiple mathematical reasoning benchmarks demonstrate that SAGE significantly accelerates convergence and outperforms static baselines.
  arXiv  Detail & Related papers  (2026-02-01T12:56:10Z)
• Denoising diffusion networks for normative modeling in neuroimaging [1.0195618602298684]
  Most neuroimaging pipelines fit one model per imaging-derived phenotype (IDP)<n>We propose denoising diffusion probabilistic models (DDPMs) as a unified conditional density estimator for IDPs.<n>We evaluate on a synthetic benchmark with heteroscedastic and multimodal age effects and on UK Biobank FreeSurfer phenotypes, scaling from dimension of 2 to 200.
  arXiv  Detail & Related papers  (2026-01-24T06:19:10Z)
• Diffusion-Driven High-Dimensional Variable Selection [6.993247097440294]
  We propose a resample-aggregate framework that exploits diffusion models' ability to generate high-fidelity synthetic data.<n>We show that the proposed method is selection consistent under mild assumptions.<n>Our method advances variable selection methodology and broadens the toolkit for interpretable, statistically rigorous analysis.
  arXiv  Detail & Related papers  (2025-08-19T14:54:20Z)
• A Distributionally-Robust Framework for Nuisance in Causal Effect Estimation [1.3053649021965603]
  Causal inference requires evaluating models on balanced distributions between treatment and control groups.<n>Most conventional statistical methods address this distribution shift through inverse probability weighting (IPW)<n>We decompose the error generalization to isolate these issues--propensity ambiguity and statistical instability--and address them through an adversarial loss function.
  arXiv  Detail & Related papers  (2025-05-23T10:34:28Z)
• Assumption-Lean Post-Integrated Inference with Surrogate Control Outcomes [6.448728765953916]
  We introduce a robust post-integrated inference (PII) method that adjusts for latent heterogeneity using control outcomes.<n>We develop semiparametric inference on projected direct effect estimands, accounting for hidden mediators, confounders, and moderators.<n>The proposed doubly robust estimators are consistent and efficient under minimal assumptions and potential misspecification.
  arXiv  Detail & Related papers  (2024-10-07T12:52:38Z)
• The Risk of Federated Learning to Skew Fine-Tuning Features and Underperform Out-of-Distribution Robustness [50.52507648690234]
  Federated learning has the risk of skewing fine-tuning features and compromising the robustness of the model. We introduce three robustness indicators and conduct experiments across diverse robust datasets. Our approach markedly enhances the robustness across diverse scenarios, encompassing various parameter-efficient fine-tuning methods.
  arXiv  Detail & Related papers  (2024-01-25T09:18:51Z)
• Quantifying predictive uncertainty of aphasia severity in stroke patients with sparse heteroscedastic Bayesian high-dimensional regression [47.1405366895538]
  Sparse linear regression methods for high-dimensional data commonly assume that residuals have constant variance, which can be violated in practice. This paper proposes estimating high-dimensional heteroscedastic linear regression models using a heteroscedastic partitioned empirical Bayes Expectation Conditional Maximization algorithm.
  arXiv  Detail & Related papers  (2023-09-15T22:06:29Z)
• Multiplicative noise and heavy tails in stochastic optimization [62.993432503309485]
  empirical optimization is central to modern machine learning, but its role in its success is still unclear. We show that it commonly arises in parameters of discrete multiplicative noise due to variance. A detailed analysis is conducted in which we describe on key factors, including recent step size, and data, all exhibit similar results on state-of-the-art neural network models.
  arXiv  Detail & Related papers  (2020-06-11T09:58:01Z)
• Instability, Computational Efficiency and Statistical Accuracy [101.32305022521024]
  We develop a framework that yields statistical accuracy based on interplay between the deterministic convergence rate of the algorithm at the population level, and its degree of (instability) when applied to an empirical object based on $n$ samples. We provide applications of our general results to several concrete classes of models, including Gaussian mixture estimation, non-linear regression models, and informative non-response models.
  arXiv  Detail & Related papers  (2020-05-22T22:30:52Z)
• Repulsive Mixture Models of Exponential Family PCA for Clustering [127.90219303669006]
  The mixture extension of exponential family principal component analysis ( EPCA) was designed to encode much more structural information about data distribution than the traditional EPCA. The traditional mixture of local EPCAs has the problem of model redundancy, i.e., overlaps among mixing components, which may cause ambiguity for data clustering. In this paper, a repulsiveness-encouraging prior is introduced among mixing components and a diversified EPCA mixture (DEPCAM) model is developed in the Bayesian framework.
  arXiv  Detail & Related papers  (2020-04-07T04:07:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.