Image Interpolation with Score-based Riemannian Metrics of Diffusion Models

• URL: http://arxiv.org/abs/2504.20288v1
• Date: Mon, 28 Apr 2025 22:04:20 GMT
• Title: Image Interpolation with Score-based Riemannian Metrics of Diffusion Models
• Authors: Shinnosuke Saito, Takashi Matsubara,
• Abstract summary: This paper introduces a novel framework that treats the data space of pre-trained diffusion models as a Riemannian manifold.<n> Experiments with MNIST and Stable Diffusion show that this geometry-aware approach yields images that are more realistic, less noisy, and more faithful to prompts than existing methods.
• Score: 9.514940899499752
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion models excel in content generation by implicitly learning the data manifold, yet they lack a practical method to leverage this manifold - unlike other deep generative models equipped with latent spaces. This paper introduces a novel framework that treats the data space of pre-trained diffusion models as a Riemannian manifold, with a metric derived from the score function. Experiments with MNIST and Stable Diffusion show that this geometry-aware approach yields image interpolations that are more realistic, less noisy, and more faithful to prompts than existing methods, demonstrating its potential for improved content generation and editing.

Related papers

• Continuous Diffusion Model for Language Modeling [57.396578974401734]
  Existing continuous diffusion models for discrete data have limited performance compared to discrete approaches.<n>We propose a continuous diffusion model for language modeling that incorporates the geometry of the underlying categorical distribution.
  arXiv  Detail & Related papers  (2025-02-17T08:54:29Z)
• Diffusion Models Learn Low-Dimensional Distributions via Subspace Clustering [15.326641037243006]
  diffusion models can effectively learn the image distribution and generate new samples. We provide theoretical insights into this phenomenon by leveraging key empirical observations. We show that the minimal number of samples required to learn the underlying distribution scales linearly with the intrinsic dimensions.
  arXiv  Detail & Related papers  (2024-09-04T04:14:02Z)
• Scaling Riemannian Diffusion Models [68.52820280448991]
  We show that our method enables us to scale to high dimensional tasks on nontrivial manifold. We model QCD densities on $SU(n)$ lattices and contrastively learned embeddings on high dimensional hyperspheres.
  arXiv  Detail & Related papers  (2023-10-30T21:27:53Z)
• Generative Modeling on Manifolds Through Mixture of Riemannian Diffusion Processes [57.396578974401734]
  We introduce a principled framework for building a generative diffusion process on general manifold. Instead of following the denoising approach of previous diffusion models, we construct a diffusion process using a mixture of bridge processes. We develop a geometric understanding of the mixture process, deriving the drift as a weighted mean of tangent directions to the data points.
  arXiv  Detail & Related papers  (2023-10-11T06:04:40Z)
• Mirror Diffusion Models for Constrained and Watermarked Generation [41.27274841596343]
  Mirror Diffusion Models (MDM) is a new class of diffusion models that generate data on convex constrained sets without losing tractability. For safety and privacy purposes, we also explore constrained sets as a new mechanism to embed invisible but quantitative information in generated data. Our work brings new algorithmic opportunities for learning tractable diffusion on complex domains.
  arXiv  Detail & Related papers  (2023-10-02T14:26:31Z)
• VTAE: Variational Transformer Autoencoder with Manifolds Learning [144.0546653941249]
  Deep generative models have demonstrated successful applications in learning non-linear data distributions through a number of latent variables. The nonlinearity of the generator implies that the latent space shows an unsatisfactory projection of the data space, which results in poor representation learning. We show that geodesics and accurate computation can substantially improve the performance of deep generative models.
  arXiv  Detail & Related papers  (2023-04-03T13:13:19Z)
• Riemannian Diffusion Models [11.306081315276089]
  Diffusion models are recent state-of-the-art methods for image generation and likelihood estimation. In this work, we generalize continuous-time diffusion models to arbitrary Riemannian manifold. Our proposed method achieves new state-of-the-art likelihoods on all benchmarks.
  arXiv  Detail & Related papers  (2022-08-16T21:18:31Z)
• Riemannian Diffusion Schr\"odinger Bridge [56.20669989459281]
  We introduce emphRiemannian Diffusion Schr"odinger Bridge to accelerate sampling of diffusion models. We validate our proposed method on synthetic data and real Earth and climate data.
  arXiv  Detail & Related papers  (2022-07-07T00:35:04Z)
• Riemannian Score-Based Generative Modeling [56.20669989459281]
  We introduce score-based generative models (SGMs) demonstrating remarkable empirical performance. Current SGMs make the underlying assumption that the data is supported on a Euclidean manifold with flat geometry. This prevents the use of these models for applications in robotics, geoscience or protein modeling.
  arXiv  Detail & Related papers  (2022-02-06T11:57:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.