Independent Mechanism Analysis and the Manifold Hypothesis

• URL: http://arxiv.org/abs/2312.13438v1
• Date: Wed, 20 Dec 2023 21:29:00 GMT
• Title: Independent Mechanism Analysis and the Manifold Hypothesis
• Authors: Shubhangi Ghosh, Luigi Gresele, Julius von K\"ugelgen, Michel Besserve, Bernhard Sch\"olkopf
• Abstract summary: Independent Mechanism Analysis (IMA) seeks to address non-identifiability in nonlinear Independent Component Analysis (ICA) We extend IMA to settings with a larger number of mixtures that reside on a manifold embedded in a higher-dimensional than the latent space. We prove that the IMA principle is approximately satisfied with high probability when the directions along which the latent components influence the observations are chosen independently at random.
• Score: 18.84905847863953
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Independent Mechanism Analysis (IMA) seeks to address non-identifiability in nonlinear Independent Component Analysis (ICA) by assuming that the Jacobian of the mixing function has orthogonal columns. As typical in ICA, previous work focused on the case with an equal number of latent components and observed mixtures. Here, we extend IMA to settings with a larger number of mixtures that reside on a manifold embedded in a higher-dimensional than the latent space -- in line with the manifold hypothesis in representation learning. For this setting, we show that IMA still circumvents several non-identifiability issues, suggesting that it can also be a beneficial principle for higher-dimensional observations when the manifold hypothesis holds. Further, we prove that the IMA principle is approximately satisfied with high probability (increasing with the number of observed mixtures) when the directions along which the latent components influence the observations are chosen independently at random. This provides a new and rigorous statistical interpretation of IMA.

Related papers

• Exact Entanglement in the Driven Quantum Symmetric Simple Exclusion Process [0.0]
  Entanglement properties of driven quantum systems can potentially differ from the equilibrium situation due to long range coherences. We derive exact formulae for its mutual information between different subsystems in the steady state and show that it satisfies a volume law. Surprisingly, the QSSEP entanglement properties only depend on data related to its transport properties and we suspect that such a relation might hold for more general mesoscopic systems.
  arXiv  Detail & Related papers  (2023-04-21T14:37:14Z)
• Targeted Separation and Convergence with Kernel Discrepancies [61.973643031360254]
  kernel-based discrepancy measures are required to (i) separate a target P from other probability measures or (ii) control weak convergence to P. In this article we derive new sufficient and necessary conditions to ensure (i) and (ii) For MMDs on separable metric spaces, we characterize those kernels that separate Bochner embeddable measures and introduce simple conditions for separating all measures with unbounded kernels.
  arXiv  Detail & Related papers  (2022-09-26T16:41:16Z)
• Probing the Robustness of Independent Mechanism Analysis for Representation Learning [9.595894646594274]
  A recently proposed approach termed Independent Mechanism Analysis (IMA) postulates that each latent source should influence the observed mixtures independently. We show that IMA-based regularization for recovering the true sources extend to mixing functions with various degrees of violation of the IMA principle.
  arXiv  Detail & Related papers  (2022-07-13T11:51:16Z)
• Shared Independent Component Analysis for Multi-Subject Neuroimaging [107.29179765643042]
  We introduce Shared Independent Component Analysis (ShICA) that models each view as a linear transform of shared independent components contaminated by additive Gaussian noise. We show that this model is identifiable if the components are either non-Gaussian or have enough diversity in noise variances. We provide empirical evidence on fMRI and MEG datasets that ShICA yields more accurate estimation of the components than alternatives.
  arXiv  Detail & Related papers  (2021-10-26T08:54:41Z)
• Discovering Latent Causal Variables via Mechanism Sparsity: A New Principle for Nonlinear ICA [81.4991350761909]
  Independent component analysis (ICA) refers to an ensemble of methods which formalize this goal and provide estimation procedure for practical application. We show that the latent variables can be recovered up to a permutation if one regularizes the latent mechanisms to be sparse.
  arXiv  Detail & Related papers  (2021-07-21T14:22:14Z)
• Visualizing spinon Fermi surfaces with time-dependent spectroscopy [62.997667081978825]
  We propose applying time-dependent photo-emission spectroscopy, an established tool in solid state systems, in cold atom quantum simulators. We show in exact diagonalization simulations of the one-dimensional $t-J$ model that the spinons start to populate previously unoccupied states in an effective band structure. The dependence of the spectral function on the time after the pump pulse reveals collective interactions among spinons.
  arXiv  Detail & Related papers  (2021-05-27T18:00:02Z)
• Bayesian Uncertainty Estimation of Learned Variational MRI Reconstruction [63.202627467245584]
  We introduce a Bayesian variational framework to quantify the model-immanent (epistemic) uncertainty. We demonstrate that our approach yields competitive results for undersampled MRI reconstruction.
  arXiv  Detail & Related papers  (2021-02-12T18:08:14Z)
• Probing symmetries of quantum many-body systems through gap ratio statistics [0.0]
  We extend the study of the gap ratio distribution P(r) to the case where discrete symmetries are present. We present a large set of applications in many-body physics, ranging from quantum clock models and anyonic chains to periodically-driven spin systems.
  arXiv  Detail & Related papers  (2020-08-25T17:11:40Z)
• Test of Eigenstate Thermalization Hypothesis Based on Local Random Matrix Theory [4.014524824655106]
  We numerically obtain a distribution of maximum fluctuations of eigenstate expectation values for different realizations of the interactions. The ergodicity of our random matrix ensembles breaks down due to locality.
  arXiv  Detail & Related papers  (2020-05-13T15:45:13Z)
• On the complex behaviour of the density in composite quantum systems [62.997667081978825]
  We study how the probability of presence of a particle is distributed between the two parts of a composite fermionic system. We prove that it is a non-perturbative property and we find out a large/small coupling constant duality. Inspired by the proof of KAM theorem, we are able to deal with this problem by introducing a cut-off in energies that eliminates these small denominators.
  arXiv  Detail & Related papers  (2020-04-14T21:41:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.