Learning low bending and low distortion manifold embeddings

• URL: http://arxiv.org/abs/2104.13189v1
• Date: Tue, 27 Apr 2021 13:51:12 GMT
• Title: Learning low bending and low distortion manifold embeddings
• Authors: Juliane Braunsmann, Marko Rajkovi\'c, Martin Rumpf, Benedikt Wirth
• Abstract summary: The encoder provides an embedding from the input data manifold into a latent space which may then be used for further processing. In this article, the embedding into latent space is regularized via a loss function that promotes an as isometric and as flat embedding. The loss functional is computed via a Monte Carlo integration which is shown to be consistent with a geometric loss functional defined directly on the embedding map.
• Score: 1.8046244926068666
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Autoencoders are a widespread tool in machine learning to transform high-dimensional data into a lowerdimensional representation which still exhibits the essential characteristics of the input. The encoder provides an embedding from the input data manifold into a latent space which may then be used for further processing. For instance, learning interpolation on the manifold may be simplified via the new manifold representation in latent space. The efficiency of such further processing heavily depends on the regularity and structure of the embedding. In this article, the embedding into latent space is regularized via a loss function that promotes an as isometric and as flat embedding as possible. The required training data comprises pairs of nearby points on the input manifold together with their local distance and their local Frechet average. This regularity loss functional even allows to train the encoder on its own. The loss functional is computed via a Monte Carlo integration which is shown to be consistent with a geometric loss functional defined directly on the embedding map. Numerical tests are performed using image data that encodes different data manifolds. The results show that smooth manifold embeddings in latent space are obtained. These embeddings are regular enough such that interpolation between not too distant points on the manifold is well approximated by linear interpolation in latent space.

Related papers

• Compression of Structured Data with Autoencoders: Provable Benefit of Nonlinearities and Depth [83.15263499262824]
  We prove that gradient descent converges to a solution that completely disregards the sparse structure of the input. We show how to improve upon Gaussian performance for the compression of sparse data by adding a denoising function to a shallow architecture. We validate our findings on image datasets, such as CIFAR-10 and MNIST.
  arXiv  Detail & Related papers  (2024-02-07T16:32:29Z)
• Improving embedding of graphs with missing data by soft manifolds [51.425411400683565]
  The reliability of graph embeddings depends on how much the geometry of the continuous space matches the graph structure. We introduce a new class of manifold, named soft manifold, that can solve this situation. Using soft manifold for graph embedding, we can provide continuous spaces to pursue any task in data analysis over complex datasets.
  arXiv  Detail & Related papers  (2023-11-29T12:48:33Z)
• Canonical normalizing flows for manifold learning [14.377143992248222]
  We propose a canonical manifold learning flow method, where a novel objective enforces the transformation matrix to have few prominent and non-degenerate basis functions. Canonical manifold flow yields a more efficient use of the latent space, automatically generating fewer prominent and distinct dimensions to represent data.
  arXiv  Detail & Related papers  (2023-10-19T13:48:05Z)
• Convolutional Filtering on Sampled Manifolds [122.06927400759021]
  We show that convolutional filtering on a sampled manifold converges to continuous manifold filtering. Our findings are further demonstrated empirically on a problem of navigation control.
  arXiv  Detail & Related papers  (2022-11-20T19:09:50Z)
• Semi-Supervised Manifold Learning with Complexity Decoupled Chart Autoencoders [65.2511270059236]
  This work introduces a chart autoencoder with an asymmetric encoding-decoding process that can incorporate additional semi-supervised information such as class labels. We discuss the theoretical approximation power of such networks that essentially depends on the intrinsic dimension of the data manifold.
  arXiv  Detail & Related papers  (2022-08-22T19:58:03Z)
• Convergent autoencoder approximation of low bending and low distortion manifold embeddings [5.5711773076846365]
  We propose and analyze a novel regularization for learning the encoder component of an autoencoder. The loss functional is computed via Monte Carlo integration with different sampling strategies for pairs of points on the input manifold. Our main theorem identifies a loss functional of the embedding map as the $Gamma$-limit of the sampling-dependent loss functionals.
  arXiv  Detail & Related papers  (2022-08-22T10:31:31Z)
• Inferring Manifolds From Noisy Data Using Gaussian Processes [17.166283428199634]
  Most existing manifold learning algorithms replace the original data with lower dimensional coordinates. This article proposes a new methodology for addressing these problems, allowing the estimated manifold between fitted data points.
  arXiv  Detail & Related papers  (2021-10-14T15:50:38Z)
• Adaptive Machine Learning for Time-Varying Systems: Low Dimensional Latent Space Tuning [91.3755431537592]
  We present a recently developed method of adaptive machine learning for time-varying systems. Our approach is to map very high (N>100k) dimensional inputs into the low dimensional (N2) latent space at the output of the encoder section of an encoder-decoder CNN. This method allows us to learn correlations within and to track their evolution in real time based on feedback without interrupts.
  arXiv  Detail & Related papers  (2021-07-13T16:05:28Z)
• Autoencoder Image Interpolation by Shaping the Latent Space [12.482988592988868]
  Autoencoders represent an effective approach for computing the underlying factors characterizing datasets of different types. We propose a regularization technique that shapes the latent representation to follow a manifold consistent with the training images.
  arXiv  Detail & Related papers  (2020-08-04T12:32:54Z)
• Manifold Learning via Manifold Deflation [105.7418091051558]
  dimensionality reduction methods provide a valuable means to visualize and interpret high-dimensional data. Many popular methods can fail dramatically, even on simple two-dimensional Manifolds. This paper presents an embedding method for a novel, incremental tangent space estimator that incorporates global structure as coordinates. Empirically, we show our algorithm recovers novel and interesting embeddings on real-world and synthetic datasets.
  arXiv  Detail & Related papers  (2020-07-07T10:04:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.