Related papers: Riemannian Diffusion Models

Riemannian Diffusion Models

URL: http://arxiv.org/abs/2208.07949v1
Date: Tue, 16 Aug 2022 21:18:31 GMT
Title: Riemannian Diffusion Models
Authors: Chin-Wei Huang, Milad Aghajohari, Avishek Joey Bose, Prakash Panangaden, Aaron Courville
Abstract summary: Diffusion models are recent state-of-the-art methods for image generation and likelihood estimation. In this work, we generalize continuous-time diffusion models to arbitrary Riemannian manifold. Our proposed method achieves new state-of-the-art likelihoods on all benchmarks.
Score: 11.306081315276089
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: Diffusion models are recent state-of-the-art methods for image generation and likelihood estimation. In this work, we generalize continuous-time diffusion models to arbitrary Riemannian manifolds and derive a variational framework for likelihood estimation. Computationally, we propose new methods for computing the Riemannian divergence which is needed in the likelihood estimation. Moreover, in generalizing the Euclidean case, we prove that maximizing this variational lower-bound is equivalent to Riemannian score matching. Empirically, we demonstrate the expressive power of Riemannian diffusion models on a wide spectrum of smooth manifolds, such as spheres, tori, hyperboloids, and orthogonal groups. Our proposed method achieves new state-of-the-art likelihoods on all benchmarks.

Related papers

von Mises Quasi-Processes for Bayesian Circular Regression [57.88921637944379]
We explore a family of expressive and interpretable distributions over circle-valued random functions. The resulting probability model has connections with continuous spin models in statistical physics. For posterior inference, we introduce a new Stratonovich-like augmentation that lends itself to fast Markov Chain Monte Carlo sampling.
arXiv Detail & Related papers (2024-06-19T01:57:21Z)
Scaling Riemannian Diffusion Models [68.52820280448991]
We show that our method enables us to scale to high dimensional tasks on nontrivial manifold. We model QCD densities on $SU(n)$ lattices and contrastively learned embeddings on high dimensional hyperspheres.
arXiv Detail & Related papers (2023-10-30T21:27:53Z)
Generative Modeling on Manifolds Through Mixture of Riemannian Diffusion Processes [57.396578974401734]
We introduce a principled framework for building a generative diffusion process on general manifold. Instead of following the denoising approach of previous diffusion models, we construct a diffusion process using a mixture of bridge processes. We develop a geometric understanding of the mixture process, deriving the drift as a weighted mean of tangent directions to the data points.
arXiv Detail & Related papers (2023-10-11T06:04:40Z)
Metropolis Sampling for Constrained Diffusion Models [11.488860260925504]
Denoising diffusion models have recently emerged as the predominant paradigm for generative modelling on image domains. We introduce an alternative, simple noretisation scheme based on the reflected Brownian motion.
arXiv Detail & Related papers (2023-07-11T17:05:23Z)
Geometric Neural Diffusion Processes [55.891428654434634]
We extend the framework of diffusion models to incorporate a series of geometric priors in infinite-dimension modelling. We show that with these conditions, the generative functional model admits the same symmetry.
arXiv Detail & Related papers (2023-07-11T16:51:38Z)
Riemannian Diffusion Schr\"odinger Bridge [56.20669989459281]
We introduce emphRiemannian Diffusion Schr"odinger Bridge to accelerate sampling of diffusion models. We validate our proposed method on synthetic data and real Earth and climate data.
arXiv Detail & Related papers (2022-07-07T00:35:04Z)
Riemannian Score-Based Generative Modeling [56.20669989459281]
We introduce score-based generative models (SGMs) demonstrating remarkable empirical performance. Current SGMs make the underlying assumption that the data is supported on a Euclidean manifold with flat geometry. This prevents the use of these models for applications in robotics, geoscience or protein modeling.
arXiv Detail & Related papers (2022-02-06T11:57:39Z)
A Riemannian Newton Trust-Region Method for Fitting Gaussian Mixture Models [0.0]
We introduce a formula for the Riemannian Hessian for Gaussian Mixture Models. On top, we propose a new Newton Trust-Region method which outperforms current approaches both in terms of runtime and number of iterations.
arXiv Detail & Related papers (2021-04-30T12:48:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.