Heavy-Tailed Principle Component Analysis

• URL: http://arxiv.org/abs/2603.11308v1
• Date: Wed, 11 Mar 2026 21:07:04 GMT
• Title: Heavy-Tailed Principle Component Analysis
• Abstract summary: We study PCA for high-dimensional data generated according to a superstatistical dependent model of the form $mathbfX = A1/2mathbfG$.<n>We formulate PCA under a logarithmic loss, which remains well defined even when moments do not exist.<n>Our main theoretical result shows that, under this loss, the principal components of the heavy-tailed observations coincide with those obtained by applying standard PCA.
• Score: 0.8866338330056278
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Principal Component Analysis (PCA) is a cornerstone of dimensionality reduction, yet its classical formulation relies critically on second-order moments and is therefore fragile in the presence of heavy-tailed data and impulsive noise. While numerous robust PCA variants have been proposed, most either assume finite variance, rely on sparsity-driven decompositions, or address robustness through surrogate loss functions without a unified treatment of infinite-variance models. In this paper, we study PCA for high-dimensional data generated according to a superstatistical dependent model of the form $\mathbf{X} = A^{1/2}\mathbf{G}$, where $A$ is a positive random scalar and $\mathbf{G}$ is a Gaussian vector. This framework captures a wide class of heavy-tailed distributions, including multivariate $t$ and sub-Gaussian $α$-stable laws. We formulate PCA under a logarithmic loss, which remains well defined even when moments do not exist. Our main theoretical result shows that, under this loss, the principal components of the heavy-tailed observations coincide with those obtained by applying standard PCA to the covariance matrix of the underlying Gaussian generator. Building on this insight, we propose robust estimators for this covariance matrix directly from heavy-tailed data and compare them with the empirical covariance and Tyler's scatter estimator. Extensive experiments, including background denoising tasks, demonstrate that the proposed approach reliably recovers principal directions and significantly outperforms classical PCA in the presence of heavy-tailed and impulsive noise, while remaining competitive under Gaussian noise.

Related papers

• Shape Operator PCA: Curvature-Aware Projections for Geometric Machine Learning [52.452902154360565]
  SHOPCA (Shape Operator-based Principal Component Analysis) is a novel method for unsupervised metric learning and dimensionality reduction.<n>We evaluate SHOPCA on more than 50 real-world benchmark datasets.
  arXiv  Detail & Related papers  (2026-08-15T16:37:47Z)
• On the Computational Complexity of Performative Prediction [59.584063949469645]
  We show that computing an $$-performatively stable point is PPAD-complete.<n>One of our key technical contributions is to extend this PPAD-hardness result to general convex domains.
  arXiv  Detail & Related papers  (2026-01-28T02:26:35Z)
• The $φ$-PCA Framework: A Unified and Efficiency-Preserving Approach with Robust Variants [0.0]
  We introduce the $phi$-PCA framework which provides a unified formulation of robust and distributed PCA.<n>We show that the partition-aggregation principle underlying $phi$-PCA offers a general strategy for developing robust and efficiency-preserving methodologies.
  arXiv  Detail & Related papers  (2025-10-15T05:21:11Z)
• T-Rex: Fitting a Robust Factor Model via Expectation-Maximization [0.0]
  We propose a novel expectation-maximization (EM) algorithm for robustly fitting statistical factor models.<n>Our approach is based on Tyler's M-estimator of the scatter matrix for an elliptical distribution.<n>We present numerical experiments on both synthetic and real examples.
  arXiv  Detail & Related papers  (2025-05-17T18:53:06Z)
• Sparse PCA with Oracle Property [115.72363972222622]
  We propose a family of estimators based on the semidefinite relaxation of sparse PCA with novel regularizations. We prove that, another estimator within the family achieves a sharper statistical rate of convergence than the standard semidefinite relaxation of sparse PCA.
  arXiv  Detail & Related papers  (2023-12-28T02:52:54Z)
• Conformal inference for regression on Riemannian Manifolds [45.560812800359685]
  We investigate prediction sets for regression scenarios when the response variable, denoted by $Y$, resides in a manifold, and the covariable, denoted by $X$, lies in an Euclidean space.<n>We prove the almost sure convergence of the empirical version of these regions on the manifold to their population counterparts.
  arXiv  Detail & Related papers  (2023-10-12T10:56:25Z)
• A Robustness Analysis of Blind Source Separation [91.3755431537592]
  Blind source separation (BSS) aims to recover an unobserved signal from its mixture $X=f(S)$ under the condition that the transformation $f$ is invertible but unknown. We present a general framework for analysing such violations and quantifying their impact on the blind recovery of $S$ from $X$. We show that a generic BSS-solution in response to general deviations from its defining structural assumptions can be profitably analysed in the form of explicit continuity guarantees.
  arXiv  Detail & Related papers  (2023-03-17T16:30:51Z)
• Bayes-optimal limits in structured PCA, and how to reach them [21.3083877172595]
  We study the paradigmatic matrix model of principal components analysis (PCA), where a rank-one matrix is corrupted by additive noise. We provide the first characterization of the Bayes-optimal limits of inference in this model. We propose a novel approximate message passing algorithm (AMP), inspired by the theory of Adaptive Thouless-Anderson-Palmer equations.
  arXiv  Detail & Related papers  (2022-10-03T21:31:41Z)
• Entrywise Recovery Guarantees for Sparse PCA via Sparsistent Algorithms [9.112172220055431]
  We provide entrywise $ell_2,infty$ bounds for Sparse PCA under a general high-dimensional subgaussian design. Our results hold for any algorithm that selects the correct support with high probability, those that are sparsistent.
  arXiv  Detail & Related papers  (2022-02-08T18:50:35Z)
• Learning to Estimate Without Bias [57.82628598276623]
  Gauss theorem states that the weighted least squares estimator is a linear minimum variance unbiased estimation (MVUE) in linear models. In this paper, we take a first step towards extending this result to non linear settings via deep learning with bias constraints. A second motivation to BCE is in applications where multiple estimates of the same unknown are averaged for improved performance.
  arXiv  Detail & Related papers  (2021-10-24T10:23:51Z)
• Enhanced Principal Component Analysis under A Collaborative-Robust Framework [89.28334359066258]
  We introduce a general collaborative-robust weight learning framework that combines weight learning and robust loss in a non-trivial way. Under the proposed framework, only a part of well-fitting samples are activated which indicates more importance during training, and others, whose errors are large, will not be ignored. In particular, the negative effects of inactivated samples are alleviated by the robust loss function.
  arXiv  Detail & Related papers  (2021-03-22T15:17:37Z)
• Robust Principal Component Analysis: A Median of Means Approach [17.446104539598895]
  Principal Component Analysis is a tool for data visualization, denoising, and dimensionality reduction. Recent supervised learning methods have shown great success in dealing with outlying observations. This paper proposes a PCA procedure based on the MoM principle.
  arXiv  Detail & Related papers  (2021-02-05T19:59:05Z)
• An $\ell_p$ theory of PCA and spectral clustering [23.90245234027558]
  Principal Component Analysis is a powerful tool in statistics and machine learning. In this paper, we develop an $ell_p$ theory for a hollowed version of PCA in Hilbert spaces. For contextual community detection, the $ell_p$ theory leads to a simple spectral algorithm that achieves the information threshold for exact recovery.
  arXiv  Detail & Related papers  (2020-06-24T21:30:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.