Mean-Shift PCA by Knockoff Mean

• URL: http://arxiv.org/abs/2605.25460v1
• Date: Mon, 25 May 2026 06:10:22 GMT
• Title: Mean-Shift PCA by Knockoff Mean
• Abstract summary: We show how to eliminate mean-shift noisy components from PCA.<n>Standard PCA is highly sensitive to shifts in the sample mean.<n>We propose a simple two-stage PCA algorithm by adding mean that identifies and removes the mean-shift component.
• Score: 1.867072656545946
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Removing noise is difficult, but adding noise is easy. In this work, we show how to eliminate mean-shift noisy components from PCA by deliberately introducing knockoff mean-shift perturbation. Standard PCA is highly sensitive to shifts in the sample mean: a small fraction of samples from a shifted distribution can cause large deviations in the leading principal components. In high-dimensional regimes, existing Robust PCA approaches cannot handle the mean-shift contamination structure inherent in the mixture model. Using tools from Random Matrix Theory, we prove that the mean-shift spikes are spectrally separable from the stable eigenvalues of the original covariance. Furthermore, the original eigenspace remains asymptotically invariant to the contamination, independent of the mixture weight. Exploiting this spectral stability, we propose a simple, two-stage PCA algorithm by adding knockoff mean that identifies and removes the mean-shift component using only standard PCA operations.

Related papers

• Diagonal Attenuation: A Finite-Sample Correction for PCA [4.568037304404448]
  Principal component analysis can rotate away from its population target when a covariance matrix is estimated from limited data.<n>We introduce diagonal attenuation, which preserves sample cross-covariances while reducing coordinatewise sample variances.<n>Across local image patches, speech spectra, and smartphone acceleration, both mask-derived and direct attenuation improve PCA under two fitting-sample budgets.
  arXiv  Detail & Related papers  (2026-09-05T01:16:07Z)
• AnchorMoE: Interpretable Time Series Classification via Anchor-Routed MoE [51.994752158953084]
  AnchorMoE is an interpretable-by-construction classification framework.<n>It encodes multi-view representations of local patches and routes them to specialized experts.<n>Experiments on real-world and synthetic benchmarks demonstrate that AnchorMoE achieves highly competitive classification performance.
  arXiv  Detail & Related papers  (2026-06-02T13:30:54Z)
• Scale-Calibrated Median-of-Means for Robust Distributed Principal Component Analysis [0.31727619150610836]
  We study a scale-calibrated median-of-means estimator for distributed principal component analysis.<n>A node-level PCA expansion shows that the mean component has the usual linear influence, whereas the subspace component is an eigengap spatial perturbation.<n>We propose robust block-scale and inference-optimal calibration rules, establish high-probability median-of-means bounds, characterize factorwise bad-node influence, and prove node-bootstrap validity.
  arXiv  Detail & Related papers  (2026-05-20T03:48:31Z)
• Measuring and Decomposing Mode Separation via the Canonical Diffusion [0.0]
  Mode separation is a fundamental geometric property of densities, difficult to quantify in high dimensions.<n>We measure mode separation through a single process intrinsic to the density.<n>We apply our framework to three settings: (i) synthetic Gaussian mixtures, where SSA tracks mutual information; (ii) SDXL text-to-image generations, where SSA and DA capture structure that entropy and PCA miss; and (iii) molecular dynamics of alanine dipeptide, where DA recovers the known slow backbones from static samples alone.
  arXiv  Detail & Related papers  (2026-05-09T08:05:55Z)
• Heavy-Tailed Principle Component Analysis [0.8866338330056278]
  We study PCA for high-dimensional data generated according to a superstatistical dependent model of the form $mathbfX = A1/2mathbfG$.<n>We formulate PCA under a logarithmic loss, which remains well defined even when moments do not exist.<n>Our main theoretical result shows that, under this loss, the principal components of the heavy-tailed observations coincide with those obtained by applying standard PCA.
  arXiv  Detail & Related papers  (2026-03-11T21:07:04Z)
• The Malignant Tail: Spectral Segregation of Label Noise in Over-Parameterized Networks [0.0]
  We experimentally isolate the Malignant Tail, a failure mode where networks functionally segregate signal and noise.<n>We show that untrained networks actively segregate noise, allowing post-hoc Explicit Spectral Truncation to surgically prune the noise-dominated subspace.<n>Our findings suggest that under label noise, excess spectral capacity is not harmless redundancy but a latent structural liability.
  arXiv  Detail & Related papers  (2026-03-02T16:39:42Z)
• Stability and Generalization of Push-Sum Based Decentralized Optimization over Directed Graphs [55.77845440440496]
  Push-based decentralized communication enables optimization over communication networks, where information exchange may be asymmetric.<n>We develop a unified uniform-stability framework for the Gradient Push (SGP) algorithm.<n>A key technical ingredient is an imbalance-aware generalization bound through two quantities.
  arXiv  Detail & Related papers  (2026-02-24T05:32:03Z)
• PCA++: How Uniformity Induces Robustness to Background Noise in Contrastive Learning [12.061533430519702]
  High-dimensional data often contain low-dimensional signals obscured by structured background noise.<n>Motivated by contrastive learning, we address the problem of recovering shared signal subspaces from positive pairs, paired observations sharing the same signal but differing in background.<n>Our baseline, PCA+, uses alignment-only contrastive learning and succeeds when background variation is mild, but fails under strong noise or high-dimensional regimes.<n>We introduce PCA++, a hard-constrained contrastive PCA that enforces identity covariance on projected features.
  arXiv  Detail & Related papers  (2025-11-15T16:11:39Z)
• Stable Neighbor Denoising for Source-free Domain Adaptive Segmentation [91.83820250747935]
  Pseudo-label noise is mainly contained in unstable samples in which predictions of most pixels undergo significant variations during self-training. We introduce the Stable Neighbor Denoising (SND) approach, which effectively discovers highly correlated stable and unstable samples. SND consistently outperforms state-of-the-art methods in various SFUDA semantic segmentation settings.
  arXiv  Detail & Related papers  (2024-06-10T21:44:52Z)
• Robust Control for Dynamical Systems With Non-Gaussian Noise via Formal Abstractions [59.605246463200736]
  We present a novel controller synthesis method that does not rely on any explicit representation of the noise distributions. First, we abstract the continuous control system into a finite-state model that captures noise by probabilistic transitions between discrete states. We use state-of-the-art verification techniques to provide guarantees on the interval Markov decision process and compute a controller for which these guarantees carry over to the original control system.
  arXiv  Detail & Related papers  (2023-01-04T10:40:30Z)
• Sampling-Based Robust Control of Autonomous Systems with Non-Gaussian Noise [59.47042225257565]
  We present a novel planning method that does not rely on any explicit representation of the noise distributions. First, we abstract the continuous system into a discrete-state model that captures noise by probabilistic transitions between states. We capture these bounds in the transition probability intervals of a so-called interval Markov decision process (iMDP)
  arXiv  Detail & Related papers  (2021-10-25T06:18:55Z)
• Enhanced Principal Component Analysis under A Collaborative-Robust Framework [89.28334359066258]
  We introduce a general collaborative-robust weight learning framework that combines weight learning and robust loss in a non-trivial way. Under the proposed framework, only a part of well-fitting samples are activated which indicates more importance during training, and others, whose errors are large, will not be ignored. In particular, the negative effects of inactivated samples are alleviated by the robust loss function.
  arXiv  Detail & Related papers  (2021-03-22T15:17:37Z)
• Suppressing Mislabeled Data via Grouping and Self-Attention [60.14212694011875]
  Deep networks achieve excellent results on large-scale clean data but degrade significantly when learning from noisy labels. This paper proposes a conceptually simple yet efficient training block, termed as Attentive Feature Mixup (AFM) It allows paying more attention to clean samples and less to mislabeled ones via sample interactions in small groups.
  arXiv  Detail & Related papers  (2020-10-29T13:54:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.