A Diffusion Model Translator for Efficient Image-to-Image Translation

• URL: http://arxiv.org/abs/2502.00307v1
• Date: Sat, 01 Feb 2025 04:01:24 GMT
• Title: A Diffusion Model Translator for Efficient Image-to-Image Translation
• Authors: Mengfei Xia, Yu Zhou, Ran Yi, Yong-Jin Liu, Wenping Wang,
• Abstract summary: We propose an efficient method that equips a diffusion model with a lightweight translator, dubbed a Diffusion Model Translator (DMT) We evaluate our approach on a range of I2I applications, including image stylization, image colorization, segmentation to image, and sketch to image, to validate its efficacy and general utility.
• Score: 60.86381807306705
• License:
• Abstract: Applying diffusion models to image-to-image translation (I2I) has recently received increasing attention due to its practical applications. Previous attempts inject information from the source image into each denoising step for an iterative refinement, thus resulting in a time-consuming implementation. We propose an efficient method that equips a diffusion model with a lightweight translator, dubbed a Diffusion Model Translator (DMT), to accomplish I2I. Specifically, we first offer theoretical justification that in employing the pioneering DDPM work for the I2I task, it is both feasible and sufficient to transfer the distribution from one domain to another only at some intermediate step. We further observe that the translation performance highly depends on the chosen timestep for domain transfer, and therefore propose a practical strategy to automatically select an appropriate timestep for a given task. We evaluate our approach on a range of I2I applications, including image stylization, image colorization, segmentation to image, and sketch to image, to validate its efficacy and general utility. The comparisons show that our DMT surpasses existing methods in both quality and efficiency. Code will be made publicly available.

Related papers

• I2I-Galip: Unsupervised Medical Image Translation Using Generative Adversarial CLIP [30.506544165999564]
  Unpaired image-to-image translation is a challenging task due to the absence of paired examples. We propose a new image-to-image translation framework named Image-to-Image-Generative-Adversarial-CLIP (I2I-Galip)
  arXiv  Detail & Related papers  (2024-09-19T01:44:50Z)
• Regularized Distribution Matching Distillation for One-step Unpaired Image-to-Image Translation [1.8434042562191815]
  We introduce Regularized Distribution Matching Distillation, applicable to unpaired image-to-image (I2I) problems. We demonstrate its empirical performance in application to several translation tasks, including 2D examples and I2I between different image datasets.
  arXiv  Detail & Related papers  (2024-06-20T22:22:31Z)
• Rethinking Score Distillation as a Bridge Between Image Distributions [97.27476302077545]
  We show that our method seeks to transport corrupted images (source) to the natural image distribution (target) Our method can be easily applied across many domains, matching or beating the performance of specialized methods. We demonstrate its utility in text-to-2D, text-based NeRF optimization, translating paintings to real images, optical illusion generation, and 3D sketch-to-real.
  arXiv  Detail & Related papers  (2024-06-13T17:59:58Z)
• Improving Diffusion-based Image Translation using Asymmetric Gradient Guidance [51.188396199083336]
  We present an approach that guides the reverse process of diffusion sampling by applying asymmetric gradient guidance. Our model's adaptability allows it to be implemented with both image-fusion and latent-dif models. Experiments show that our method outperforms various state-of-the-art models in image translation tasks.
  arXiv  Detail & Related papers  (2023-06-07T12:56:56Z)
• Wuerstchen: An Efficient Architecture for Large-Scale Text-to-Image Diffusion Models [6.821399706256863]
  W"urstchen is a novel architecture for text-to-image synthesis that combines competitive performance with unprecedented cost-effectiveness. A key contribution of our work is to develop a latent diffusion technique in which we learn a detailed but extremely compact semantic image representation.
  arXiv  Detail & Related papers  (2023-06-01T13:00:53Z)
• BBDM: Image-to-image Translation with Brownian Bridge Diffusion Models [50.39417112077254]
  A novel image-to-image translation method based on the Brownian Bridge Diffusion Model (BBDM) is proposed. To the best of our knowledge, it is the first work that proposes Brownian Bridge diffusion process for image-to-image translation.
  arXiv  Detail & Related papers  (2022-05-16T13:47:02Z)
• Dual Diffusion Implicit Bridges for Image-to-Image Translation [104.59371476415566]
  Common image-to-image translation methods rely on joint training over data from both source and target domains. We present Dual Diffusion Implicit Bridges (DDIBs), an image translation method based on diffusion models. DDIBs allow translations between arbitrary pairs of source-target domains, given independently trained diffusion models on respective domains.
  arXiv  Detail & Related papers  (2022-03-16T04:10:45Z)
• Unsupervised Image-to-Image Translation via Pre-trained StyleGAN2 Network [73.5062435623908]
  We propose a new I2I translation method that generates a new model in the target domain via a series of model transformations. By feeding the latent vector into the generated model, we can perform I2I translation between the source domain and target domain.
  arXiv  Detail & Related papers  (2020-10-12T13:51:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.