BBDM: Image-to-image Translation with Brownian Bridge Diffusion Models

• URL: http://arxiv.org/abs/2205.07680v2
• Date: Thu, 23 Mar 2023 08:29:12 GMT
• Title: BBDM: Image-to-image Translation with Brownian Bridge Diffusion Models
• Authors: Bo Li, Kaitao Xue, Bin Liu, Yu-Kun Lai
• Abstract summary: A novel image-to-image translation method based on the Brownian Bridge Diffusion Model (BBDM) is proposed. To the best of our knowledge, it is the first work that proposes Brownian Bridge diffusion process for image-to-image translation.
• Score: 50.39417112077254
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image-to-image translation is an important and challenging problem in computer vision and image processing. Diffusion models (DM) have shown great potentials for high-quality image synthesis, and have gained competitive performance on the task of image-to-image translation. However, most of the existing diffusion models treat image-to-image translation as conditional generation processes, and suffer heavily from the gap between distinct domains. In this paper, a novel image-to-image translation method based on the Brownian Bridge Diffusion Model (BBDM) is proposed, which models image-to-image translation as a stochastic Brownian bridge process, and learns the translation between two domains directly through the bidirectional diffusion process rather than a conditional generation process. To the best of our knowledge, it is the first work that proposes Brownian Bridge diffusion process for image-to-image translation. Experimental results on various benchmarks demonstrate that the proposed BBDM model achieves competitive performance through both visual inspection and measurable metrics.

Related papers

• EBDM: Exemplar-guided Image Translation with Brownian-bridge Diffusion Models [42.55874233756394]
  We propose a novel approach termed Exemplar-guided Image Translation with Brownian-Bridge Diffusion Models (EBDM) Our method formulates the task as a Brownian bridge process, a diffusion process with a fixed initial point as structure control and translates into the corresponding photo-realistic image while being conditioned solely on the given exemplar image.
  arXiv  Detail & Related papers  (2024-10-13T11:10:34Z)
• Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model [101.65105730838346]
  We introduce Transfusion, a recipe for training a multi-modal model over discrete and continuous data. We pretrain multiple Transfusion models up to 7B parameters from scratch on a mixture of text and image data. Our experiments show that Transfusion scales significantly better than quantizing images and training a language model over discrete image tokens.
  arXiv  Detail & Related papers  (2024-08-20T17:48:20Z)
• Improving Diffusion-based Image Translation using Asymmetric Gradient Guidance [51.188396199083336]
  We present an approach that guides the reverse process of diffusion sampling by applying asymmetric gradient guidance. Our model's adaptability allows it to be implemented with both image-fusion and latent-dif models. Experiments show that our method outperforms various state-of-the-art models in image translation tasks.
  arXiv  Detail & Related papers  (2023-06-07T12:56:56Z)
• LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models [62.75006608940132]
  This work proposes to enhance prompt understanding capabilities in text-to-image diffusion models. Our method leverages a pretrained large language model for grounded generation in a novel two-stage process. Our method significantly outperforms the base diffusion model and several strong baselines in accurately generating images.
  arXiv  Detail & Related papers  (2023-05-23T03:59:06Z)
• Dual Diffusion Implicit Bridges for Image-to-Image Translation [104.59371476415566]
  Common image-to-image translation methods rely on joint training over data from both source and target domains. We present Dual Diffusion Implicit Bridges (DDIBs), an image translation method based on diffusion models. DDIBs allow translations between arbitrary pairs of source-target domains, given independently trained diffusion models on respective domains.
  arXiv  Detail & Related papers  (2022-03-16T04:10:45Z)
• UNIT-DDPM: UNpaired Image Translation with Denoising Diffusion Probabilistic Models [19.499490172426427]
  We propose a novel unpaired image-to-image translation method that uses denoising diffusion probabilistic models without requiring adversarial training. Our method, UNpaired Image Translation with Denoising Diffusion Probabilistic Models (UNIT-DDPM), trains a generative model to infer the joint distribution of images over both domains as a Markov chain.
  arXiv  Detail & Related papers  (2021-04-12T11:22:56Z)
• Unsupervised Image-to-Image Translation via Pre-trained StyleGAN2 Network [73.5062435623908]
  We propose a new I2I translation method that generates a new model in the target domain via a series of model transformations. By feeding the latent vector into the generated model, we can perform I2I translation between the source domain and target domain.
  arXiv  Detail & Related papers  (2020-10-12T13:51:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.