EBDM: Exemplar-guided Image Translation with Brownian-bridge Diffusion Models

• URL: http://arxiv.org/abs/2410.09802v1
• Date: Sun, 13 Oct 2024 11:10:34 GMT
• Title: EBDM: Exemplar-guided Image Translation with Brownian-bridge Diffusion Models
• Authors: Eungbean Lee, Somi Jeong, Kwanghoon Sohn,
• Abstract summary: We propose a novel approach termed Exemplar-guided Image Translation with Brownian-Bridge Diffusion Models (EBDM) Our method formulates the task as a Brownian bridge process, a diffusion process with a fixed initial point as structure control and translates into the corresponding photo-realistic image while being conditioned solely on the given exemplar image.
• Score: 42.55874233756394
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Exemplar-guided image translation, synthesizing photo-realistic images that conform to both structural control and style exemplars, is attracting attention due to its ability to enhance user control over style manipulation. Previous methodologies have predominantly depended on establishing dense correspondences across cross-domain inputs. Despite these efforts, they incur quadratic memory and computational costs for establishing dense correspondence, resulting in limited versatility and performance degradation. In this paper, we propose a novel approach termed Exemplar-guided Image Translation with Brownian-Bridge Diffusion Models (EBDM). Our method formulates the task as a stochastic Brownian bridge process, a diffusion process with a fixed initial point as structure control and translates into the corresponding photo-realistic image while being conditioned solely on the given exemplar image. To efficiently guide the diffusion process toward the style of exemplar, we delineate three pivotal components: the Global Encoder, the Exemplar Network, and the Exemplar Attention Module to incorporate global and detailed texture information from exemplar images. Leveraging Bridge diffusion, the network can translate images from structure control while exclusively conditioned on the exemplar style, leading to more robust training and inference processes. We illustrate the superiority of our method over competing approaches through comprehensive benchmark evaluations and visual results.

Related papers

• Diffusion-Based Conditional Image Editing through Optimized Inference with Guidance [46.922018440110826]
  We present a training-free approach for text-driven image-to-image translation based on a pretrained text-to-image diffusion model. Our method achieves outstanding image-to-image translation performance on various tasks when combined with the pretrained Stable Diffusion model.
  arXiv  Detail & Related papers  (2024-12-20T11:15:31Z)
• Oscillation Inversion: Understand the structure of Large Flow Model through the Lens of Inversion Method [60.88467353578118]
  We show that a fixed-point-inspired iterative approach to invert real-world images does not achieve convergence, instead oscillating between distinct clusters. We introduce a simple and fast distribution transfer technique that facilitates image enhancement, stroke-based recoloring, as well as visual prompt-guided image editing.
  arXiv  Detail & Related papers  (2024-11-17T17:45:37Z)
• Improving Diffusion-based Image Translation using Asymmetric Gradient Guidance [51.188396199083336]
  We present an approach that guides the reverse process of diffusion sampling by applying asymmetric gradient guidance. Our model's adaptability allows it to be implemented with both image-fusion and latent-dif models. Experiments show that our method outperforms various state-of-the-art models in image translation tasks.
  arXiv  Detail & Related papers  (2023-06-07T12:56:56Z)
• Paint by Example: Exemplar-based Image Editing with Diffusion Models [35.84464684227222]
  In this paper, we investigate exemplar-guided image editing for more precise control. We achieve this goal by leveraging self-supervised training to disentangle and re-organize the source image and the exemplar. We demonstrate that our method achieves an impressive performance and enables controllable editing on in-the-wild images with high fidelity.
  arXiv  Detail & Related papers  (2022-11-23T18:59:52Z)
• Pretraining is All You Need for Image-to-Image Translation [59.43151345732397]
  We propose to use pretraining to boost general image-to-image translation. We show that the proposed pretraining-based image-to-image translation (PITI) is capable of synthesizing images of unprecedented realism and faithfulness.
  arXiv  Detail & Related papers  (2022-05-25T17:58:26Z)
• BBDM: Image-to-image Translation with Brownian Bridge Diffusion Models [50.39417112077254]
  A novel image-to-image translation method based on the Brownian Bridge Diffusion Model (BBDM) is proposed. To the best of our knowledge, it is the first work that proposes Brownian Bridge diffusion process for image-to-image translation.
  arXiv  Detail & Related papers  (2022-05-16T13:47:02Z)
• TSIT: A Simple and Versatile Framework for Image-to-Image Translation [103.92203013154403]
  We introduce a simple and versatile framework for image-to-image translation. We provide a carefully designed two-stream generative model with newly proposed feature transformations. This allows multi-scale semantic structure information and style representation to be effectively captured and fused by the network. A systematic study compares the proposed method with several state-of-the-art task-specific baselines, verifying its effectiveness in both perceptual quality and quantitative evaluations.
  arXiv  Detail & Related papers  (2020-07-23T15:34:06Z)
• PerceptionGAN: Real-world Image Construction from Provided Text through Perceptual Understanding [11.985768957782641]
  We propose a method to provide good images by incorporating perceptual understanding in the discriminator module. We show that the perceptual information included in the initial image is improved while modeling image distribution at multiple stages. More importantly, the proposed method can be integrated into the pipeline of other state-of-the-art text-based-image-generation models.
  arXiv  Detail & Related papers  (2020-07-02T09:23:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.