Beyond Explained Variance: A Cautionary Tale of PCA

• URL: http://arxiv.org/abs/2605.13520v2
• Date: Sun, 17 May 2026 13:40:28 GMT
• Title: Beyond Explained Variance: A Cautionary Tale of PCA
• Abstract summary: We propose a generative probabilistic-geometric model in which the data are sampled uniformly from a unit circle.<n>Under this model, pairwise cosine distances follow an arcsine distribution, in qualitative agreement with the observed U-shaped distribution, thereby independently supporting the analysis based on t-SNE and persistent homology.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We address shortcomings of principal component analysis (PCA) for visualizing high-dimensional data lying on a nonlinear low-dimensional manifold via two-dimensional scatterplots, focusing on a fossil teeth dataset from the early mammalian insectivore Kuehneotherium. While the PCA scatterplot reported by Jolliffe and Cadima (Philosophical Transactions of the Royal Society A, 2016) shows clustering in the region where PC2 < 0, our analysis based on t-SNE and persistent homology (PH) reveals a ring-like structure with no evident clustering and intrinsic dimensionality equal to one. We further propose a generative probabilistic-geometric model in which the data are sampled uniformly from a unit circle. Under this model, pairwise cosine distances follow an arcsine distribution, in qualitative agreement with the observed U-shaped distribution, thereby independently supporting the analysis based on t-SNE and persistent homology.

Related papers

• A Theory of Speciation in Generative Diffusion Models on Compact Riemannian Manifolds [0.05729426778193397]
  Speciation in generative diffusion models denotes the emergence of distinct stable branches during denoising.<n>We go beyond existing theoretical descriptions, which usually identify speciation with a symmetric pitchfork bifurcation.<n>A chart-based intrinsic score-learning scheme based on neural networks contrasts the theoretically predicted transitions on prototypal.
  arXiv  Detail & Related papers  (2026-08-24T20:03:31Z)
• Extracting central charge from ground-state overlaps of spatially deformed Hamiltonians [9.71677905891384]
  We show that the conformal anomaly of a $(1+1)$-dimensional conformal field theory can be extracted directly from a ground-state wave-function overlap.<n>Motivated by this result, we construct a lattice estimator based solely on ground-state overlaps.<n> Numerical results demonstrate that the resulting overlaps provide a simple and robust probe of the central charge in microscopic models.
  arXiv  Detail & Related papers  (2026-05-29T18:00:02Z)
• SEED: Targeted Data Selection by Weighted Independent Set [76.68391670109433]
  We develop a robust and scalable data selection pipeline dubbed SEED.<n>SEED consistently outperforms state-of-the-art methods on instruction tuning, visual instruction tuning, and semantic segmentation.
  arXiv  Detail & Related papers  (2026-05-15T07:26:54Z)
• Measuring and Decomposing Mode Separation via the Canonical Diffusion [0.0]
  Mode separation is a fundamental geometric property of densities, difficult to quantify in high dimensions.<n>We measure mode separation through a single process intrinsic to the density.<n>We apply our framework to three settings: (i) synthetic Gaussian mixtures, where SSA tracks mutual information; (ii) SDXL text-to-image generations, where SSA and DA capture structure that entropy and PCA miss; and (iii) molecular dynamics of alanine dipeptide, where DA recovers the known slow backbones from static samples alone.
  arXiv  Detail & Related papers  (2026-05-09T08:05:55Z)
• Curvature-Aware PCA with Geodesic Tangent Space Aggregation for Semi-Supervised Learning [52.452902154360565]
  GTSA-PCA is a geometric extension of Principal Component Analysis.<n>It integrates curvature awareness and geodesic consistency within a unified spectral framework.<n>Our results position GTSA-PCA as a principled bridge between statistical and geometric approaches to dimensionality reduction.
  arXiv  Detail & Related papers  (2026-04-20T20:36:36Z)
• Quotient-Based Posterior Analysis for Euclidean Latent Space Models [0.5371337604556311]
  We propose a quotient-based posterior analysis for Euclidean latent space models.<n>We show how coherent posterior analysis can reveal latent relational structure beyond a single embedding.
  arXiv  Detail & Related papers  (2026-04-03T05:21:10Z)
• Convergence of Score-Based Discrete Diffusion Models: A Discrete-Time Analysis [56.442307356162864]
  We study the theoretical aspects of score-based discrete diffusion models under the Continuous Time Markov Chain (CTMC) framework.<n>We introduce a discrete-time sampling algorithm in the general state space $[S]d$ that utilizes score estimators at predefined time points.<n>Our convergence analysis employs a Girsanov-based method and establishes key properties of the discrete score function.
  arXiv  Detail & Related papers  (2024-10-03T09:07:13Z)
• Joint Bayesian Inference of Graphical Structure and Parameters with a Single Generative Flow Network [59.79008107609297]
  We propose in this paper to approximate the joint posterior over the structure of a Bayesian Network. We use a single GFlowNet whose sampling policy follows a two-phase process. Since the parameters are included in the posterior distribution, this leaves more flexibility for the local probability models.
  arXiv  Detail & Related papers  (2023-05-30T19:16:44Z)
• Wasserstein Archetypal Analysis [9.54262011088777]
  Archetypal analysis is an unsupervised machine learning method that summarizes data using a convex polytope. We consider an alternative formulation of archetypal analysis based on the Wasserstein metric.
  arXiv  Detail & Related papers  (2022-10-25T19:50:09Z)
• Locality of Spontaneous Symmetry Breaking and Universal Spacing Distribution of Topological Defects Formed Across a Phase Transition [62.997667081978825]
  A continuous phase transition results in the formation of topological defects with a density predicted by the Kibble-Zurek mechanism (KZM) We characterize the spatial distribution of point-like topological defects in the resulting nonequilibrium state and model it using a Poisson point process in arbitrary spatial dimension with KZM density.
  arXiv  Detail & Related papers  (2022-02-23T19:00:06Z)
• Topological transitions with continuously monitored free fermions [68.8204255655161]
  We show the presence of a topological phase transition that is of a different universality class than that observed in stroboscopic projective circuits. We find that this entanglement transition is well identified by a combination of the bipartite entanglement entropy and the topological entanglement entropy.
  arXiv  Detail & Related papers  (2021-12-17T22:01:54Z)
• Consistency of archetypal analysis [10.424626933990272]
  Archetypal analysis is an unsupervised learning method that uses a convex polytope to summarize multivariate data. In this paper, we prove a consistency result that shows if the data is independently sampled from a probability measure with bounded support. We also obtain the convergence rate of the optimal objective values under appropriate assumptions on the distribution.
  arXiv  Detail & Related papers  (2020-10-16T04:07:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.