On the Correlation between Random Variables and their Principal Components

• URL: http://arxiv.org/abs/2310.06139v1
• Date: Mon, 9 Oct 2023 20:35:38 GMT
• Title: On the Correlation between Random Variables and their Principal Components
• Authors: Zenon Gniazdowski
• Abstract summary: The article attempts to find an algebraic formula describing the correlation coefficients between random variables and the principal components representing them. It is possible to apply this formula to optimize the number of principal components in Principal Component Analysis, as well as to optimize the number of factors in Factor Analysis.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The article attempts to find an algebraic formula describing the correlation coefficients between random variables and the principal components representing them. As a result of the analysis, starting from selected statistics relating to individual random variables, the equivalents of these statistics relating to a set of random variables were presented in the language of linear algebra, using the concepts of vector and matrix. This made it possible, in subsequent steps, to derive the expected formula. The formula found is identical to the formula used in Factor Analysis to calculate factor loadings. The discussion showed that it is possible to apply this formula to optimize the number of principal components in Principal Component Analysis, as well as to optimize the number of factors in Factor Analysis.

Related papers

• Spectral decomposition-assisted multi-study factor analysis [7.925272817108244]
  Methods are applied to integrate three studies on gene associations among immune cells. Conditional distribution of factor loadings has a simple product form across outcomes.
  arXiv  Detail & Related papers  (2025-02-20T14:33:40Z)
• Variance-based sensitivity analysis in the presence of correlated input variables [0.0]
  We propose an extension of the classical Sobol' estimator for the estimation of variance based sensitivity indices. The approach assumes a linear correlation model which is used to decompose the contribution of an input variable into a correlated and an uncorrelated part.
  arXiv  Detail & Related papers  (2024-08-09T08:32:58Z)
• Relaxation Fluctuations of Correlation Functions: Spin and Random Matrix Models [0.0]
  We study the fluctuation average and variance of certain correlation functions as a diagnostic measure of quantum chaos. We identify the three distinct phases of the models: the ergodic, the fractal, and the localized phases.
  arXiv  Detail & Related papers  (2024-07-31T14:45:46Z)
• Scaling and renormalization in high-dimensional regression [72.59731158970894]
  This paper presents a succinct derivation of the training and generalization performance of a variety of high-dimensional ridge regression models. We provide an introduction and review of recent results on these topics, aimed at readers with backgrounds in physics and deep learning.
  arXiv  Detail & Related papers  (2024-05-01T15:59:00Z)
• Conformal inference for regression on Riemannian Manifolds [49.7719149179179]
  We investigate prediction sets for regression scenarios when the response variable, denoted by $Y$, resides in a manifold, and the covariable, denoted by X, lies in Euclidean space. We prove the almost sure convergence of the empirical version of these regions on the manifold to their population counterparts.
  arXiv  Detail & Related papers  (2023-10-12T10:56:25Z)
• A Heavy-Tailed Algebra for Probabilistic Programming [53.32246823168763]
  We propose a systematic approach for analyzing the tails of random variables. We show how this approach can be used during the static analysis (before drawing samples) pass of a probabilistic programming language compiler. Our empirical results confirm that inference algorithms that leverage our heavy-tailed algebra attain superior performance across a number of density modeling and variational inference tasks.
  arXiv  Detail & Related papers  (2023-06-15T16:37:36Z)
• HiPerformer: Hierarchically Permutation-Equivariant Transformer for Time Series Forecasting [56.95572957863576]
  We propose a hierarchically permutation-equivariant model that considers both the relationship among components in the same group and the relationship among groups. The experiments conducted on real-world data demonstrate that the proposed method outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-14T05:11:52Z)
• Data-Driven Influence Functions for Optimization-Based Causal Inference [105.5385525290466]
  We study a constructive algorithm that approximates Gateaux derivatives for statistical functionals by finite differencing. We study the case where probability distributions are not known a priori but need to be estimated from data.
  arXiv  Detail & Related papers  (2022-08-29T16:16:22Z)
• Principal Component Analysis versus Factor Analysis [0.0]
  The article discusses selected problems related to both principal component analysis (PCA) and factor analysis (FA) A new criterion for determining the number of factors and principal components is discussed. An efficient algorithm for determining the number of factors in FA, which complies with this criterion, was also proposed.
  arXiv  Detail & Related papers  (2021-10-21T16:43:00Z)
• Eigen Analysis of Self-Attention and its Reconstruction from Partial Computation [58.80806716024701]
  We study the global structure of attention scores computed using dot-product based self-attention. We find that most of the variation among attention scores lie in a low-dimensional eigenspace. We propose to compute scores only for a partial subset of token pairs, and use them to estimate scores for the remaining pairs.
  arXiv  Detail & Related papers  (2021-06-16T14:38:42Z)
• Factoring Multidimensional Data to Create a Sophisticated Bayes Classifier [0.0]
  We derive an explicit formula for calculating the marginal likelihood of a given factorization of a categorical dataset. These likelihoods can be used to order all possible factorizations and select the "best" way to factor the overall distribution from which the dataset is drawn.
  arXiv  Detail & Related papers  (2021-05-11T16:34:12Z)
• Degenerate Gaussian factors for probabilistic inference [0.0]
  We propose a parametrised factor that enables inference on Gaussian networks where linear dependencies exist among the random variables. By using this principled factor definition, degeneracies can be accommodated accurately and automatically at little additional computational cost.
  arXiv  Detail & Related papers  (2021-04-30T13:58:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.