De-floored Principal Component Regression: When Rank Selection Alone Is Insufficient for Prediction

• URL: http://arxiv.org/abs/2607.16638v1
• Date: Sat, 18 Jul 2026 04:50:59 GMT
• Title: De-floored Principal Component Regression: When Rank Selection Alone Is Insufficient for Prediction
• Abstract summary: Principal component regression regularizes high-dimensional prediction by choosing a spectral cutoff.<n>We prove an ordinary-PCR prediction-risk lower bound uniform over all ranks and a high-probability dPCR upper bound.<n>An exact risk decomposition explains the separation: denominator inflation is governed by first spectral mass, whereas the clean prediction cost of correction is governed by squared spectral mass.
• Score: 11.327284468812948
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Principal component regression (PCR) regularizes high-dimensional prediction by choosing a spectral cutoff, but rank selection cannot correct systematic inflation of the retained empirical eigenvalues. We study clean Gaussian random designs in which the aggregate covariance tail creates a nearly scalar sample-space floor comparable to the predictive head scale. De-floored principal component regression (dPCR) retains the cutoff and subtracts an estimated floor from the retained denominators. We prove an ordinary-PCR prediction-risk lower bound uniform over all ranks and a high-probability dPCR upper bound. When the floor is sharp and inexpensive to remove in population prediction risk, the conditional risk of dPCR is asymptotically negligible relative to that of the best ordinary PCR rank. An exact risk decomposition explains the separation: denominator inflation is governed by first spectral mass, whereas the clean prediction cost of correction is governed by squared spectral mass. A same-sample trimmed-mean floor estimate attains the oracle dPCR upper-bound rate at a prespecified rank, and the separation persists under approximate predictive alignment when the tail prediction-energy fraction vanishes. Separate pointwise fixed-aspect formulas show that the risk-optimal positive scalar correction improves rank-$1$ PCR, whereas mean-floor subtraction is generally not optimal for a broad Marchenko--Pastur bulk.

Related papers

• Characterizing Heterogeneous Rates in Finite Mixture Estimation via Partial Optimal Transport [88.55403215843967]
  Voronoi-based partial optimal transport (VPOT) for obtaining refined local and global convergence guarantees for the maximum likelihood estimator of the mixing measure.<n>Results hold in arbitrary fixed dimension without requiring mixing proportions to be uniformly bounded away from zero or prior knowledge of the true number of mixture components.
  arXiv  Detail & Related papers  (2026-09-15T04:32:53Z)
• Diagonal Attenuation: A Finite-Sample Correction for PCA [4.568037304404448]
  Principal component analysis can rotate away from its population target when a covariance matrix is estimated from limited data.<n>We introduce diagonal attenuation, which preserves sample cross-covariances while reducing coordinatewise sample variances.<n>Across local image patches, speech spectra, and smartphone acceleration, both mask-derived and direct attenuation improve PCA under two fitting-sample budgets.
  arXiv  Detail & Related papers  (2026-09-05T01:16:07Z)
• When Proxy Prediction Becomes Equation Reconstruction: Diagnostics and Residual Learning for Factor-Derived Proxy Supervision [0.0]
  High predictive accuracy may reflect reconstruction of the proxy-generating equation rather than robustness to degraded factor information.<n>We introduce a diagnostic framework that combines degraded-formula references, classical tree-based baselines, matched direct and formula-feature predictors, contextual ablations, tail-error analysis, and robustness scoring.<n>We then propose RASPL, a formula-preserving residual framework that retains the degraded formula estimate as the prediction anchor and learns an adaptively gated contextual correction.
  arXiv  Detail & Related papers  (2026-08-05T02:54:34Z)
• Prediction-Only Distillation in Linear and Logistic Regression [61.07540493350384]
  Self-distillation is typically studied when the student is retrained on the teacher's original training inputs.<n>We study SD in this prediction-only regime through a fresh-X prediction-mixed scheme.<n>For binary logistic regression, we show that prediction mixing can outperform both the teacher and the pure-distilled.
  arXiv  Detail & Related papers  (2026-07-16T20:45:56Z)
• PIT-SUN: A Deployable Empirical Marginal Transform Framework with Expectation-Consistent Recovery for Regression in Recommender Systems [17.665661827812205]
  Estimating original-space conditional expectations is central to value-driven recommender systems.<n>Standard MSE is expectation-consistent in principle, but its gradients become unstable on heavy-tailed, zero-inflated, and multimodal targets.<n>We propose PIT-SUN, a deployable empirical marginal recovery framework.
  arXiv  Detail & Related papers  (2026-07-09T08:00:06Z)
• DARC: Disagreement-Aware Alignment via Risk-Constrained Decoding [59.16244104797919]
  We propose **Disagreement-Aware Alignment via Risk-Constrained Decoding (DARC)**.<n>DARC frames response selection as distributionally robust, risk-sensitive decision making.<n>Experiments on alignment benchmarks show that DARC reduces disagreement and tail risk while maintaining competitive average quality.
  arXiv  Detail & Related papers  (2026-03-09T09:21:29Z)
• Distribution-informed Efficient Conformal Prediction for Full Ranking [22.380815981596403]
  Quantifying uncertainty is critical for the safe deployment of ranking models in real-world applications.<n>Recent work offers a rigorous solution using conformal prediction in a full ranking scenario, which aims to construct prediction sets for the absolute ranks of test items.<n>We propose Distribution-informed Conformal Ranking (DCR), which produces efficient prediction sets by deriving the exact distribution of non-conformity scores.
  arXiv  Detail & Related papers  (2026-01-30T16:16:44Z)
• Beyond Uncertainty Sets: Leveraging Optimal Transport to Extend Conformal Predictive Distribution to Multivariate Settings [0.14504054468850666]
  Conformal prediction (CP) constructs uncertainty sets for model outputs with finite-sample coverage guarantees.<n>We show that optimal assignment is piecewise-constant across a fixed polyhedral partition of the score space.<n>This allows us to characterize the entire prediction set tractably, and provides the machinery to address a deeper limitation of prediction sets.
  arXiv  Detail & Related papers  (2025-11-19T05:59:01Z)
• Calibrated Principal Component Regression [5.890015449615021]
  Principal Component Regression (PCR) incurs truncation bias when the true regression vector has mass outside the retained principal components (PC)<n>We propose Calibrated Principal Component Regression (CPCR), which first learns a low-variance prior in the PC subspace and then calibrates the model in the original feature space via a centered Tikhonov step.
  arXiv  Detail & Related papers  (2025-10-21T19:01:42Z)
• From Noisy Traces to Stable Gradients: Bias-Variance Optimized Preference Optimization for Aligning Large Reasoning Models [90.45197506653341]
  Large reasoning models generate intermediate reasoning traces before producing final answers.<n> aligning LRMs with human preferences, a crucial prerequisite for model deployment, remains underexplored.<n>A common workaround optimized a single sampled trajectory, which introduces substantial gradient variance from trace sampling.
  arXiv  Detail & Related papers  (2025-10-06T17:58:01Z)
• Sparse PCA with Oracle Property [115.72363972222622]
  We propose a family of estimators based on the semidefinite relaxation of sparse PCA with novel regularizations. We prove that, another estimator within the family achieves a sharper statistical rate of convergence than the standard semidefinite relaxation of sparse PCA.
  arXiv  Detail & Related papers  (2023-12-28T02:52:54Z)
• Quantifying predictive uncertainty of aphasia severity in stroke patients with sparse heteroscedastic Bayesian high-dimensional regression [47.1405366895538]
  Sparse linear regression methods for high-dimensional data commonly assume that residuals have constant variance, which can be violated in practice. This paper proposes estimating high-dimensional heteroscedastic linear regression models using a heteroscedastic partitioned empirical Bayes Expectation Conditional Maximization algorithm.
  arXiv  Detail & Related papers  (2023-09-15T22:06:29Z)
• Benign Overfitting of Constant-Stepsize SGD for Linear Regression [122.70478935214128]
  inductive biases are central in preventing overfitting empirically. This work considers this issue in arguably the most basic setting: constant-stepsize SGD for linear regression. We reflect on a number of notable differences between the algorithmic regularization afforded by (unregularized) SGD in comparison to ordinary least squares.
  arXiv  Detail & Related papers  (2021-03-23T17:15:53Z)
• On Model Identification and Out-of-Sample Prediction of Principal Component Regression: Applications to Synthetic Controls [20.96904429337912]
  We analyze principal component regression (PCR) in a high-dimensional error-in-variables setting with fixed design. We establish non-asymptotic out-of-sample prediction guarantees that improve upon the best known rates.
  arXiv  Detail & Related papers  (2020-10-27T17:07:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.