GeoGuide: Geometric guidance of diffusion models

• URL: http://arxiv.org/abs/2407.12889v1
• Date: Wed, 17 Jul 2024 07:56:27 GMT
• Title: GeoGuide: Geometric guidance of diffusion models
• Authors: Mateusz Poleski, Jacek Tabor, Przemysław Spurek,
• Abstract summary: GeoGuide is a guidance model based on tracing the distance of the diffusion model's trajectory from the data manifold. It surpasses the probabilistic approach ADM-G with respect to both the FID scores and the quality of the generated images.
• Score: 8.34616719984217
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion models are among the most effective methods for image generation. This is in particular because, unlike GANs, they can be easily conditioned during training to produce elements with desired class or properties. However, guiding a pre-trained diffusion model to generate elements from previously unlabeled data is significantly more challenging. One of the possible solutions was given by the ADM-G guiding approach. Although ADM-G successfully generates elements from the given class, there is a significant quality gap compared to a model originally conditioned on this class. In particular, the FID score obtained by the ADM-G-guided diffusion model is nearly three times lower than the class-conditioned guidance. We demonstrate that this issue is partly due to ADM-G providing minimal guidance during the final stage of the denoising process. To address this problem, we propose GeoGuide, a guidance model based on tracing the distance of the diffusion model's trajectory from the data manifold. The main idea of GeoGuide is to produce normalized adjustments during the backward denoising process. As shown in the experiments, GeoGuide surpasses the probabilistic approach ADM-G with respect to both the FID scores and the quality of the generated images.

Related papers

• IFH: a Diffusion Framework for Flexible Design of Graph Generative Models [53.219279193440734]
  Graph generative models can be classified into two prominent families: one-shot models, which generate a graph in one go, and sequential models, which generate a graph by successive additions of nodes and edges. This paper proposes a graph generative model, called Insert-Fill-Halt (IFH), that supports the specification of a sequentiality degree.
  arXiv  Detail & Related papers  (2024-08-23T16:24:40Z)
• DGInStyle: Domain-Generalizable Semantic Segmentation with Image Diffusion Models and Stylized Semantic Control [68.14798033899955]
  Large, pretrained latent diffusion models (LDMs) have demonstrated an extraordinary ability to generate creative content. However, are they usable as large-scale data generators, e.g., to improve tasks in the perception stack, like semantic segmentation? We investigate this question in the context of autonomous driving, and answer it with a resounding "yes"
  arXiv  Detail & Related papers  (2023-12-05T18:34:12Z)
• Semi-Implicit Denoising Diffusion Models (SIDDMs) [50.30163684539586]
  Existing models such as Denoising Diffusion Probabilistic Models (DDPM) deliver high-quality, diverse samples but are slowed by an inherently high number of iterative steps. We introduce a novel approach that tackles the problem by matching implicit and explicit factors. We demonstrate that our proposed method obtains comparable generative performance to diffusion-based models and vastly superior results to models with a small number of sampling steps.
  arXiv  Detail & Related papers  (2023-06-21T18:49:22Z)
• Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models [77.83923746319498]
  We propose a framework called Diff-Instruct to instruct the training of arbitrary generative models. We show that Diff-Instruct results in state-of-the-art single-step diffusion-based models. Experiments on refining GAN models show that the Diff-Instruct can consistently improve the pre-trained generators of GAN models.
  arXiv  Detail & Related papers  (2023-05-29T04:22:57Z)
• Deep Manifold Learning with Graph Mining [80.84145791017968]
  We propose a novel graph deep model with a non-gradient decision layer for graph mining. The proposed model has achieved state-of-the-art performance compared to the current models.
  arXiv  Detail & Related papers  (2022-07-18T04:34:08Z)
• Riemannian Score-Based Generative Modeling [56.20669989459281]
  We introduce score-based generative models (SGMs) demonstrating remarkable empirical performance. Current SGMs make the underlying assumption that the data is supported on a Euclidean manifold with flat geometry. This prevents the use of these models for applications in robotics, geoscience or protein modeling.
  arXiv  Detail & Related papers  (2022-02-06T11:57:39Z)
• PLACE dropout: A Progressive Layer-wise and Channel-wise Dropout for Domain Generalization [29.824723021053565]
  Domain generalization (DG) aims to learn a generic model from multiple observed source domains. The major challenge in DG is that the model inevitably faces a severe overfitting issue due to the domain gap between source and target domains. We develop a novel layer-wise and channel-wise dropout for DG, which randomly selects one layer and then randomly selects its channels to conduct dropout.
  arXiv  Detail & Related papers  (2021-12-07T13:23:52Z)
• Guided Integrated Gradients: An Adaptive Path Method for Removing Noise [9.792727625917083]
  Integrated Gradients (IG) is a commonly used feature attribution method for deep neural networks. We show that one of the causes of the problem is the accumulation of noise along the IG path. We propose adapting the attribution path itself -- conditioning the path not just on the image but also on the model being explained.
  arXiv  Detail & Related papers  (2021-06-17T20:00:55Z)
• Scalable Semi-supervised Landmark Localization for X-ray Images using Few-shot Deep Adaptive Graph [19.588348005574165]
  Based on a fully-supervised graph-based method, DAG, we proposed a semi-supervised extension of it, termed few-shot DAG. It first trains a DAG model on the labeled data and then fine-tunes the pre-trained model on the unlabeled data with a teacher-student SSL mechanism. We extensively evaluated our method on pelvis, hand and chest landmark detection tasks.
  arXiv  Detail & Related papers  (2021-04-29T19:46:18Z)
• Demonstrating the Evolution of GANs through t-SNE [0.4588028371034407]
  Evolutionary algorithms, such as COEGAN, were recently proposed as a solution to improve the GAN training. In this work, we propose an evaluation method based on t-distributed Neighbour Embedding (t-SNE) to assess the progress of GANs. A metric based on the resulting t-SNE maps and the Jaccard index is proposed to represent the model quality.
  arXiv  Detail & Related papers  (2021-01-31T20:07:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.