Foundations of Independent Component Analysis

• URL: http://arxiv.org/abs/2608.13229v1
• Date: Thu, 13 Aug 2026 13:31:26 GMT
• Title: Foundations of Independent Component Analysis
• Abstract summary: We develop the theory of the characteristic functions of probability measures on $mathbbRd$.<n>We show that the independent sources are identifiable up to translation, permutation, scales and signs.<n>We present the online equivariant gradient descent ICA algorithm for recovering the independent sources from data.
• Score: 16.08983109077834
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present the mathematical foundations of linear independent component analysis (ICA) models based on standard literature in a self-contained note. It is aimed at readers with a background in measure-theoretic probability theory. We first develop the theory of the characteristic functions of probability measures on $\mathbb{R}^d$, including their analyticity and the way in which they determine and characterise the distributions. We then focus on several identifiability results of ICA models with successively strengthened assumptions on the sources: from merely non-constant, to non-Gaussian, to Gaussian-free independent sources. Under the strictest assumptions, we show that the independent sources are identifiable up to translation, permutation, scales and signs, and this even in the presence of additive Gaussian noise. Furthermore, we present the online equivariant gradient descent ICA algorithm for recovering the independent sources from data, in the standard complete noiseless non-Gaussian ICA setting.

Related papers

• Linear Independent Component Analysis via Optimal Transport [16.109077391631917]
  OT-ICA can be used for applied ICA tasks without distributional assumptions.<n> OT-ICA can be used for applied ICA tasks without distributional assumptions.
  arXiv  Detail & Related papers  (2026-07-15T17:56:11Z)
• Contrast-Free ICA and Causal Inference via Wasserstein Distances to the Gaussian [2.495377652912602]
  We study the squared $2$-Wasserstein distance to the standard Gaussian as a non-Gaussianity criterion.<n>We use it for linear Independent Component Analysis (ICA) and causal discovery in Linear Non-Gaussian Acyclic Models (LiNGAM)
  arXiv  Detail & Related papers  (2026-07-14T14:49:31Z)
• The Procrustean Bed of Time Series: The Optimization Bias of Point-wise Loss [53.542743390809356]
  This paper aims to provide a first-principles analysis of the Expectation of Optimization Bias (EOB)<n>Our analysis reveals a fundamental paradigm paradox: the more deterministic and structured the time series, the more severe the bias by point-wise loss function.<n>We present a concrete solution that simultaneously achieves both principles via DFT or DWT.
  arXiv  Detail & Related papers  (2025-12-21T06:08:22Z)
• Beyond independent component analysis: identifiability and algorithms [0.8793721044482612]
  We show that pairwise mean independence answers the question of how much one can relax independence.<n>Our results apply to distributions with the required zero pattern at any cumulant tensor.
  arXiv  Detail & Related papers  (2025-10-08T20:38:56Z)
• On the Identifiability of Sparse ICA without Assuming Non-Gaussianity [20.333908367541895]
  We develop an identifiability theory that relies on second-order statistics without imposing further preconditions on the distribution of sources. We propose two estimation methods based on second-order statistics and sparsity constraint.
  arXiv  Detail & Related papers  (2024-08-19T18:51:42Z)
• Large-Sample Properties of Non-Stationary Source Separation for Gaussian Signals [2.2557806157585834]
  We develop large-sample theory for NSS-JD, a popular method of non-stationary source separation. We show that the consistency of the unmixing estimator and its convergence to a limiting Gaussian distribution at the standard square root rate are shown to hold. Simulation experiments are used to verify the theoretical results and to study the impact of block length on the separation.
  arXiv  Detail & Related papers  (2022-09-21T08:13:20Z)
• On the Identifiability of Nonlinear ICA: Sparsity and Beyond [20.644375143901488]
  How to make the nonlinear ICA model identifiable up to certain trivial indeterminacies is a long-standing problem in unsupervised learning. Recent breakthroughs reformulate the standard independence assumption of sources as conditional independence given some auxiliary variables. We show that under specific instantiations of such constraints, the independent latent sources can be identified from their nonlinear mixtures up to a permutation.
  arXiv  Detail & Related papers  (2022-06-15T18:24:22Z)
• Nonlinear Independent Component Analysis for Continuous-Time Signals [85.59763606620938]
  We study the classical problem of recovering a multidimensional source process from observations of mixtures of this process. We show that this recovery is possible for many popular models of processes (up to order and monotone scaling of their coordinates) if the mixture is given by a sufficiently differentiable, invertible function.
  arXiv  Detail & Related papers  (2021-02-04T20:28:44Z)
• Disentangling Observed Causal Effects from Latent Confounders using Method of Moments [67.27068846108047]
  We provide guarantees on identifiability and learnability under mild assumptions. We develop efficient algorithms based on coupled tensor decomposition with linear constraints to obtain scalable and guaranteed solutions.
  arXiv  Detail & Related papers  (2021-01-17T07:48:45Z)
• Stochastic Approximation for Online Tensorial Independent Component Analysis [98.34292831923335]
  Independent component analysis (ICA) has been a popular dimension reduction tool in statistical machine learning and signal processing. In this paper, we present a by-product online tensorial algorithm that estimates for each independent component.
  arXiv  Detail & Related papers  (2020-12-28T18:52:37Z)
• Nonlinear ISA with Auxiliary Variables for Learning Speech Representations [51.9516685516144]
  We introduce a theoretical framework for nonlinear Independent Subspace Analysis (ISA) in the presence of auxiliary variables. We propose an algorithm that learns unsupervised speech representations whose subspaces are independent.
  arXiv  Detail & Related papers  (2020-07-25T14:53:09Z)
• Nonparametric Score Estimators [49.42469547970041]
  Estimating the score from a set of samples generated by an unknown distribution is a fundamental task in inference and learning of probabilistic models. We provide a unifying view of these estimators under the framework of regularized nonparametric regression. We propose score estimators based on iterative regularization that enjoy computational benefits from curl-free kernels and fast convergence.
  arXiv  Detail & Related papers  (2020-05-20T15:01:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.