Topological Obstructions and How to Avoid Them

• URL: http://arxiv.org/abs/2312.07529v1
• Date: Tue, 12 Dec 2023 18:56:14 GMT
• Title: Topological Obstructions and How to Avoid Them
• Authors: Babak Esmaeili, Robin Walters, Heiko Zimmermann, Jan-Willem van de Meent
• Abstract summary: We show that local optima can arise due to singularities or an incorrect degree or winding number. We propose a new flow-based model that maps data points to multimodal distributions over geometric spaces.
• Score: 22.45861345237023
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Incorporating geometric inductive biases into models can aid interpretability and generalization, but encoding to a specific geometric structure can be challenging due to the imposed topological constraints. In this paper, we theoretically and empirically characterize obstructions to training encoders with geometric latent spaces. We show that local optima can arise due to singularities (e.g. self-intersection) or due to an incorrect degree or winding number. We then discuss how normalizing flows can potentially circumvent these obstructions by defining multimodal variational distributions. Inspired by this observation, we propose a new flow-based model that maps data points to multimodal distributions over geometric spaces and empirically evaluate our model on 2 domains. We observe improved stability during training and a higher chance of converging to a homeomorphic encoder.

Related papers

• Geometric Trajectory Diffusion Models [58.853975433383326]
  Generative models have shown great promise in generating 3D geometric systems. Existing approaches only operate on static structures, neglecting the fact that physical systems are always dynamic in nature. We propose geometric trajectory diffusion models (GeoTDM), the first diffusion model for modeling the temporal distribution of 3D geometric trajectories.
  arXiv  Detail & Related papers  (2024-10-16T20:36:41Z)
• Decoder ensembling for learned latent geometries [15.484595752241122]
  We show how to easily compute geodesics on the associated expected manifold. We find this simple and reliable, thereby coming one step closer to easy-to-use latent geometries.
  arXiv  Detail & Related papers  (2024-08-14T12:35:41Z)
• Geometric Neural Diffusion Processes [55.891428654434634]
  We extend the framework of diffusion models to incorporate a series of geometric priors in infinite-dimension modelling. We show that with these conditions, the generative functional model admits the same symmetry.
  arXiv  Detail & Related papers  (2023-07-11T16:51:38Z)
• Latent Traversals in Generative Models as Potential Flows [113.4232528843775]
  We propose to model latent structures with a learned dynamic potential landscape. Inspired by physics, optimal transport, and neuroscience, these potential landscapes are learned as physically realistic partial differential equations. Our method achieves both more qualitatively and quantitatively disentangled trajectories than state-of-the-art baselines.
  arXiv  Detail & Related papers  (2023-04-25T15:53:45Z)
• Geometric Scattering on Measure Spaces [12.0756034112778]
  We introduce a general, unified model for geometric scattering on measure spaces. We consider finite measure spaces that are obtained from randomly sampling an unknown manifold. We propose two methods for constructing a data-driven graph on which the associated graph scattering transform approximates the scattering transform on the underlying manifold.
  arXiv  Detail & Related papers  (2022-08-17T22:40:09Z)
• Moser Flow: Divergence-based Generative Modeling on Manifolds [49.04974733536027]
  Moser Flow (MF) is a new class of generative models within the family of continuous normalizing flows (CNF) MF does not require invoking or backpropagating through an ODE solver during training. We demonstrate for the first time the use of flow models for sampling from general curved surfaces.
  arXiv  Detail & Related papers  (2021-08-18T09:00:24Z)
• A Unifying and Canonical Description of Measure-Preserving Diffusions [60.59592461429012]
  A complete recipe of measure-preserving diffusions in Euclidean space was recently derived unifying several MCMC algorithms into a single framework. We develop a geometric theory that improves and generalises this construction to any manifold.
  arXiv  Detail & Related papers  (2021-05-06T17:36:55Z)
• Continuous normalizing flows on manifolds [0.342658286826597]
  We describe how the recently introduced Neural ODEs and continuous normalizing flows can be extended to arbitrary smooth manifold. We propose a general methodology for parameterizing vector fields on these spaces and demonstrate how gradient-based learning can be performed.
  arXiv  Detail & Related papers  (2021-03-14T15:35:19Z)
• GELATO: Geometrically Enriched Latent Model for Offline Reinforcement Learning [54.291331971813364]
  offline reinforcement learning approaches can be divided into proximal and uncertainty-aware methods. In this work, we demonstrate the benefit of combining the two in a latent variational model. Our proposed metrics measure both the quality of out of distribution samples as well as the discrepancy of examples in the data.
  arXiv  Detail & Related papers  (2021-02-22T19:42:40Z)
• Generative Model without Prior Distribution Matching [26.91643368299913]
  Variational Autoencoder (VAE) and its variations are classic generative models by learning a low-dimensional latent representation to satisfy some prior distribution. We propose to let the prior match the embedding distribution rather than imposing the latent variables to fit the prior.
  arXiv  Detail & Related papers  (2020-09-23T09:33:24Z)
• Neural Ordinary Differential Equations on Manifolds [0.342658286826597]
  Recently normalizing flows in Euclidean space based on Neural ODEs show great promise, yet suffer the same limitations. We show how vector fields provide a general framework for parameterizing a flexible class of invertible mapping on these spaces.
  arXiv  Detail & Related papers  (2020-06-11T17:56:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.