Diagonal Attenuation: A Finite-Sample Correction for PCA

• URL: http://arxiv.org/abs/2609.05796v1
• Date: Sat, 05 Sep 2026 01:16:07 GMT
• Title: Diagonal Attenuation: A Finite-Sample Correction for PCA
• Abstract summary: Principal component analysis can rotate away from its population target when a covariance matrix is estimated from limited data.<n>We introduce diagonal attenuation, which preserves sample cross-covariances while reducing coordinatewise sample variances.<n>Across local image patches, speech spectra, and smartphone acceleration, both mask-derived and direct attenuation improve PCA under two fitting-sample budgets.
• Score: 4.568037304404448
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Principal component analysis (PCA) can rotate away from its population target when a covariance matrix is estimated from limited data. We introduce diagonal attenuation, which preserves sample cross-covariances while reducing coordinatewise sample variances. The method is revealed exactly by averaging a linear full-output reconstruction loss over random input masks; studying the correction directly extends it beyond the range attainable by masking. We isolate the part of the random coupling between retained and omitted population directions that is contributed by sample-variance errors, and show how attenuation can reduce the resulting rotation. Under balanced marginal variances, we derive an explicit expected-risk theorem, uniform over the attenuation path for all sufficiently large finite samples, and obtain the asymptotically risk-minimizing strength. For general covariances, we characterize when attenuation leaves the population PCA subspace unchanged and give a risk theorem that also accounts for changing eigengaps and the population cost when the target moves. Simulations track this tradeoff from exact preservation back to PCA. Across local image patches, speech spectra, and smartphone acceleration, both mask-derived and direct attenuation improve PCA under two fitting-sample budgets, and one of them has the largest mean gain among seven methods in every data--budget cell. The full path selects strengths beyond the mask-derived boundary on $63\%$--$95\%$ of the subsamples.

Related papers

• Shape Operator PCA: Curvature-Aware Projections for Geometric Machine Learning [52.452902154360565]
  SHOPCA (Shape Operator-based Principal Component Analysis) is a novel method for unsupervised metric learning and dimensionality reduction.<n>We evaluate SHOPCA on more than 50 real-world benchmark datasets.
  arXiv  Detail & Related papers  (2026-08-15T16:37:47Z)
• Convex-Hull-Neighborhood Smooth Dual Generalization: Controlling Local Correction Propagation in Offline RL [7.803284875836737]
  bootstrapping can amplify estimation errors in offline reinforcement learning.<n>We propose Convex Hull Neighborhood Smooth Dual Generalization, which expresses the Bellman backup as an in-sample value target plus a CHN-local correction.<n> Experiments on Gym-MuJoCo and AntMaze show strong aggregate performance and stable value estimation.
  arXiv  Detail & Related papers  (2026-08-04T04:30:02Z)
• Mean-Shift PCA by Knockoff Mean [1.867072656545946]
  We show how to eliminate mean-shift noisy components from PCA.<n>Standard PCA is highly sensitive to shifts in the sample mean.<n>We propose a simple two-stage PCA algorithm by adding mean that identifies and removes the mean-shift component.
  arXiv  Detail & Related papers  (2026-05-25T06:10:22Z)
• The Value of Covariance Matching in Gaussian DDPMs and the Lanczos Sampler [4.538165276831438]
  A central error measure in Gaussian DDPMs is the path-space KL divergence between the exact reverse chain and the learned Gaussian reverse process.<n>We show that matching the full posterior covariance breaks this barrier, yielding an order-wise improvement that reduces the path KL to $O(1/T2)$.<n>We prove that LGS approximation error decays exponentially in the number of Lanczos steps, where each Lanczos step requires a single Jacobian-vector product.
  arXiv  Detail & Related papers  (2026-05-21T16:57:27Z)
• Closing the Approximation Gap of Partial AUC Optimization: A Tale of Two Formulations [121.39938773554523]
  The Area Under the ROC Curve (AUC) is a pivotal evaluation metric in real-world scenarios with both class imbalance and decision constraints.<n>We present two simple instance-wise minimax reformulations to close the approximation gap of PAUC optimization.<n>The resulting algorithms enjoy a linear per-iteration computational complexity w.r.t. the sample size and a convergence rate of $O(-2/3)$ for typical one-way and two-way PAUCs.
  arXiv  Detail & Related papers  (2025-12-01T02:52:33Z)
• Wasserstein-regularized Conformal Prediction under General Distribution Shift [21.29977879431466]
  Conformal prediction yields a prediction set with guaranteed $1-alpha$ coverage of the true target under the i.i.d. assumption.<n>We propose a Wasserstein distance-based upper bound of the coverage gap and analyze the bound using probability measure pushforwards.<n>We exploit the separation to design an algorithm based on importance weighting and regularized representation learning (WR-CP) to reduce the Wasserstein bound with a finite-sample error bound.
  arXiv  Detail & Related papers  (2025-01-23T07:29:44Z)
• Sparse PCA with Oracle Property [115.72363972222622]
  We propose a family of estimators based on the semidefinite relaxation of sparse PCA with novel regularizations. We prove that, another estimator within the family achieves a sharper statistical rate of convergence than the standard semidefinite relaxation of sparse PCA.
  arXiv  Detail & Related papers  (2023-12-28T02:52:54Z)
• Empirical Bayes Covariance Decomposition, and a solution to the Multiple Tuning Problem in Sparse PCA [2.5382095320488673]
  Sparse Principal Components Analysis (PCA) has been proposed as a way to improve both interpretability and reliability of PCA. We present a solution to the "multiple tuning problem" using Empirical Bayes methods.
  arXiv  Detail & Related papers  (2023-12-06T04:00:42Z)
• Adaptive Annealed Importance Sampling with Constant Rate Progress [68.8204255655161]
  Annealed Importance Sampling (AIS) synthesizes weighted samples from an intractable distribution. We propose the Constant Rate AIS algorithm and its efficient implementation for $alpha$-divergences.
  arXiv  Detail & Related papers  (2023-06-27T08:15:28Z)
• LSDAT: Low-Rank and Sparse Decomposition for Decision-based Adversarial Attack [74.5144793386864]
  LSDAT crafts perturbations in the low-dimensional subspace formed by the sparse component of the input sample and that of an adversarial sample. LSD works directly in the image pixel domain to guarantee that non-$ell$ constraints, such as sparsity, are satisfied.
  arXiv  Detail & Related papers  (2021-03-19T13:10:47Z)
• Private Stochastic Non-Convex Optimization: Adaptive Algorithms and Tighter Generalization Bounds [72.63031036770425]
  We propose differentially private (DP) algorithms for bound non-dimensional optimization. We demonstrate two popular deep learning methods on the empirical advantages over standard gradient methods.
  arXiv  Detail & Related papers  (2020-06-24T06:01:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.