Related papers: Disentangling by Subspace Diffusion

Disentangling by Subspace Diffusion

URL: http://arxiv.org/abs/2006.12982v2
Date: Wed, 18 Nov 2020 13:48:45 GMT
Title: Disentangling by Subspace Diffusion
Authors: David Pfau, Irina Higgins, Aleksandar Botev and S\'ebastien Racani\`ere
Abstract summary: We show that fully unsupervised factorization of a data manifold is possible if the true metric of the manifold is known. Our work reduces the question of whether unsupervised metric learning is possible, providing a unifying insight into the geometric nature of representation learning.
Score: 72.1895236605335
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: We present a novel nonparametric algorithm for symmetry-based disentangling of data manifolds, the Geometric Manifold Component Estimator (GEOMANCER). GEOMANCER provides a partial answer to the question posed by Higgins et al. (2018): is it possible to learn how to factorize a Lie group solely from observations of the orbit of an object it acts on? We show that fully unsupervised factorization of a data manifold is possible if the true metric of the manifold is known and each factor manifold has nontrivial holonomy -- for example, rotation in 3D. Our algorithm works by estimating the subspaces that are invariant under random walk diffusion, giving an approximation to the de Rham decomposition from differential geometry. We demonstrate the efficacy of GEOMANCER on several complex synthetic manifolds. Our work reduces the question of whether unsupervised disentangling is possible to the question of whether unsupervised metric learning is possible, providing a unifying insight into the geometric nature of representation learning.

Related papers

Scaling Riemannian Diffusion Models [68.52820280448991]
We show that our method enables us to scale to high dimensional tasks on nontrivial manifold. We model QCD densities on $SU(n)$ lattices and contrastively learned embeddings on high dimensional hyperspheres.
arXiv Detail & Related papers (2023-10-30T21:27:53Z)
A Heat Diffusion Perspective on Geodesic Preserving Dimensionality Reduction [66.21060114843202]
We propose a more general heat kernel based manifold embedding method that we call heat geodesic embeddings. Results show that our method outperforms existing state of the art in preserving ground truth manifold distances. We also showcase our method on single cell RNA-sequencing datasets with both continuum and cluster structure.
arXiv Detail & Related papers (2023-05-30T13:58:50Z)
Toward a Geometric Theory of Manifold Untangling [12.229735866373073]
We conjecture that there is a more general solution to manifold untangling in the topological space without artificially defining any distance metric. General strategies of both global manifold embedding and local manifold flattening are presented and connected with existing work on the untangling of image, audio, and language data.
arXiv Detail & Related papers (2023-03-07T19:47:01Z)
Measuring dissimilarity with diffeomorphism invariance [94.02751799024684]
We introduce DID, a pairwise dissimilarity measure applicable to a wide range of data spaces. We prove that DID enjoys properties which make it relevant for theoretical study and practical use.
arXiv Detail & Related papers (2022-02-11T13:51:30Z)
Inferring Manifolds From Noisy Data Using Gaussian Processes [17.166283428199634]
Most existing manifold learning algorithms replace the original data with lower dimensional coordinates. This article proposes a new methodology for addressing these problems, allowing the estimated manifold between fitted data points.
arXiv Detail & Related papers (2021-10-14T15:50:38Z)
A Unifying and Canonical Description of Measure-Preserving Diffusions [60.59592461429012]
A complete recipe of measure-preserving diffusions in Euclidean space was recently derived unifying several MCMC algorithms into a single framework. We develop a geometric theory that improves and generalises this construction to any manifold.
arXiv Detail & Related papers (2021-05-06T17:36:55Z)
Learning disentangled representations via product manifold projection [10.677966716893762]
We propose a novel approach to disentangle the generative factors of variation underlying a given set of observations. Our method builds upon the idea that the (unknown) low-dimensional manifold underlying the data space can be explicitly modeled as a product of submanifolds.
arXiv Detail & Related papers (2021-03-02T10:59:59Z)
Manifold Density Estimation via Generalized Dequantization [9.090451761951101]
Some kinds of data are not well-modeled by supposing that their underlying geometry is Euclidean. For instance, some kinds of data may be known to lie on the surface of a sphere. We propose a method, inspired by the literature on "dequantization," which we interpret through a coordinate transformation of an ambient Euclidean space.
arXiv Detail & Related papers (2021-02-14T12:40:41Z)
Manifold Learning via Manifold Deflation [105.7418091051558]
dimensionality reduction methods provide a valuable means to visualize and interpret high-dimensional data. Many popular methods can fail dramatically, even on simple two-dimensional Manifolds. This paper presents an embedding method for a novel, incremental tangent space estimator that incorporates global structure as coordinates. Empirically, we show our algorithm recovers novel and interesting embeddings on real-world and synthetic datasets.
arXiv Detail & Related papers (2020-07-07T10:04:28Z)
Sample complexity and effective dimension for regression on manifolds [13.774258153124205]
We consider the theory of regression on a manifold using kernel reproducing Hilbert space methods. We show that certain spaces of smooth functions on a manifold are effectively finite-dimensional, with a complexity that scales according to the manifold dimension.
arXiv Detail & Related papers (2020-06-13T14:09:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.