Semi-Supervised Manifold Learning with Complexity Decoupled Chart Autoencoders

• URL: http://arxiv.org/abs/2208.10570v2
• Date: Fri, 04 Oct 2024 16:34:17 GMT
• Title: Semi-Supervised Manifold Learning with Complexity Decoupled Chart Autoencoders
• Authors: Stefan C. Schonsheck, Scott Mahan, Timo Klock, Alexander Cloninger, Rongjie Lai,
• Abstract summary: This work introduces a chart autoencoder with an asymmetric encoding-decoding process that can incorporate additional semi-supervised information such as class labels. We discuss the approximation power of such networks and derive a bound that essentially depends on the intrinsic dimension of the data manifold rather than the dimension of ambient space.
• Score: 45.29194877564103
• License:
• Abstract: Autoencoding is a popular method in representation learning. Conventional autoencoders employ symmetric encoding-decoding procedures and a simple Euclidean latent space to detect hidden low-dimensional structures in an unsupervised way. Some modern approaches to novel data generation such as generative adversarial networks askew this symmetry, but still employ a pair of massive networks--one to generate the image and another to judge the images quality based on priors learned from a training set. This work introduces a chart autoencoder with an asymmetric encoding-decoding process that can incorporate additional semi-supervised information such as class labels. Besides enhancing the capability for handling data with complicated topological and geometric structures, the proposed model can successfully differentiate nearby but disjoint manifolds and intersecting manifolds with only a small amount of supervision. Moreover, this model only requires a low-complexity encoding operation, such as a locally defined linear projection. We discuss the approximation power of such networks and derive a bound that essentially depends on the intrinsic dimension of the data manifold rather than the dimension of ambient space. Next we incorporate bounds for the sampling rate of training data need to faithfully represent a given data manifold. We present numerical experiments that verify that the proposed model can effectively manage data with multi-class nearby but disjoint manifolds of different classes, overlapping manifolds, and manifolds with non-trivial topology. Finally, we conclude with some experiments on computer vision and molecular dynamics problems which showcase the efficacy of our methods on real-world data.

Related papers

• SpaceMesh: A Continuous Representation for Learning Manifold Surface Meshes [61.110517195874074]
  We present a scheme to directly generate manifold, polygonal meshes of complex connectivity as the output of a neural network. Our key innovation is to define a continuous latent connectivity space at each mesh, which implies the discrete mesh. In applications, this approach not only yields high-quality outputs from generative models, but also enables directly learning challenging geometry processing tasks such as mesh repair.
  arXiv  Detail & Related papers  (2024-09-30T17:59:03Z)
• On-Manifold Projected Gradient Descent [0.0]
  This work provides a computable, direct, and mathematically rigorous approximation to the differential geometry of class manifold for high-dimensional data. Tools are applied to the setting of neural network image classifiers, where we generate novel, on-manifold data samples.
  arXiv  Detail & Related papers  (2023-08-23T17:50:50Z)
• VTAE: Variational Transformer Autoencoder with Manifolds Learning [144.0546653941249]
  Deep generative models have demonstrated successful applications in learning non-linear data distributions through a number of latent variables. The nonlinearity of the generator implies that the latent space shows an unsatisfactory projection of the data space, which results in poor representation learning. We show that geodesics and accurate computation can substantially improve the performance of deep generative models.
  arXiv  Detail & Related papers  (2023-04-03T13:13:19Z)
• Unsupervised Manifold Linearizing and Clustering [19.879641608165887]
  We propose to optimize the Maximal Coding Reduction metric with respect to both the data representation and a novel doubly cluster membership. Experiments on CIFAR-10, -20, -100, and TinyImageNet-200 datasets show that the proposed method is much more accurate and scalable than state-of-the-art deep clustering methods.
  arXiv  Detail & Related papers  (2023-01-04T20:08:23Z)
• Decoupled Multi-task Learning with Cyclical Self-Regulation for Face Parsing [71.19528222206088]
  We propose a novel Decoupled Multi-task Learning with Cyclical Self-Regulation for face parsing. Specifically, DML-CSR designs a multi-task model which comprises face parsing, binary edge, and category edge detection. Our method achieves the new state-of-the-art performance on the Helen, CelebA-HQ, and LapaMask datasets.
  arXiv  Detail & Related papers  (2022-03-28T02:12:30Z)
• Geometry-Aware Hierarchical Bayesian Learning on Manifolds [5.182379239800725]
  We propose a hierarchical Bayesian learning model for learning on manifold-valued vision data. We first introduce a kernel with the properties of geometry-awareness and intra- Kernel convolution. We then use a Gaussian process regression to organize the inputs and finally implement a hierarchical Bayesian network for the feature aggregation.
  arXiv  Detail & Related papers  (2021-10-30T05:47:05Z)
• Manifold Topology Divergence: a Framework for Comparing Data Manifolds [109.0784952256104]
  We develop a framework for comparing data manifold, aimed at the evaluation of deep generative models. Based on the Cross-Barcode, we introduce the Manifold Topology Divergence score (MTop-Divergence) We demonstrate that the MTop-Divergence accurately detects various degrees of mode-dropping, intra-mode collapse, mode invention, and image disturbance.
  arXiv  Detail & Related papers  (2021-06-08T00:30:43Z)
• Category-Learning with Context-Augmented Autoencoder [63.05016513788047]
  Finding an interpretable non-redundant representation of real-world data is one of the key problems in Machine Learning. We propose a novel method of using data augmentations when training autoencoders. We train a Variational Autoencoder in such a way, that it makes transformation outcome predictable by auxiliary network.
  arXiv  Detail & Related papers  (2020-10-10T14:04:44Z)
• Extendable and invertible manifold learning with geometry regularized autoencoders [9.742277703732187]
  A fundamental task in data exploration is to extract simplified low dimensional representations that capture intrinsic geometry in data. Common approaches to this task use kernel methods for manifold learning. We present a new method for integrating both approaches by incorporating a geometric regularization term in the bottleneck of the autoencoder.
  arXiv  Detail & Related papers  (2020-07-14T15:59:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.