Conditional Consistency Guided Image Translation and Enhancement

• URL: http://arxiv.org/abs/2501.01223v2
• Date: Fri, 03 Jan 2025 17:30:10 GMT
• Title: Conditional Consistency Guided Image Translation and Enhancement
• Authors: Amil Bhagat, Milind Jain, A. V. Subramanyam,
• Abstract summary: We introduce Conditional Consistency Models ( CCMs) for multi-domain image translation. We implement these modifications by introducing task-specific conditional inputs that guide the denoising process. We evaluate CCMs on 10 different datasets demonstrating their effectiveness in producing high-quality translated images.
• Score: 0.0
• License:
• Abstract: Consistency models have emerged as a promising alternative to diffusion models, offering high-quality generative capabilities through single-step sample generation. However, their application to multi-domain image translation tasks, such as cross-modal translation and low-light image enhancement remains largely unexplored. In this paper, we introduce Conditional Consistency Models (CCMs) for multi-domain image translation by incorporating additional conditional inputs. We implement these modifications by introducing task-specific conditional inputs that guide the denoising process, ensuring that the generated outputs retain structural and contextual information from the corresponding input domain. We evaluate CCMs on 10 different datasets demonstrating their effectiveness in producing high-quality translated images across multiple domains. Code is available at https://github.com/amilbhagat/Conditional-Consistency-Models.

Related papers

• ARMADA: Attribute-Based Multimodal Data Augmentation [93.05614922383822]
  Attribute-based Multimodal Data Augmentation (ARMADA) is a novel multimodal data augmentation method via knowledge-guided manipulation of visual attributes. ARMADA is a novel multimodal data generation framework that: (i) extracts knowledge-grounded attributes from symbolic KBs for semantically consistent yet distinctive image-text pair generation. This also highlights the need to leverage external knowledge proxies for enhanced interpretability and real-world grounding.
  arXiv  Detail & Related papers  (2024-08-19T15:27:25Z)
• Img-Diff: Contrastive Data Synthesis for Multimodal Large Language Models [49.439311430360284]
  We introduce a novel data synthesis method inspired by contrastive learning and image difference captioning. Our key idea involves challenging the model to discern both matching and distinct elements. We leverage this generated dataset to fine-tune state-of-the-art (SOTA) MLLMs.
  arXiv  Detail & Related papers  (2024-08-08T17:10:16Z)
• Many-to-many Image Generation with Auto-regressive Diffusion Models [59.5041405824704]
  This paper introduces a domain-general framework for many-to-many image generation, capable of producing interrelated image series from a given set of images. We present MIS, a novel large-scale multi-image dataset, containing 12M synthetic multi-image samples, each with 25 interconnected images. We learn M2M, an autoregressive model for many-to-many generation, where each image is modeled within a diffusion framework.
  arXiv  Detail & Related papers  (2024-04-03T23:20:40Z)
• Improving Diffusion-based Image Translation using Asymmetric Gradient Guidance [51.188396199083336]
  We present an approach that guides the reverse process of diffusion sampling by applying asymmetric gradient guidance. Our model's adaptability allows it to be implemented with both image-fusion and latent-dif models. Experiments show that our method outperforms various state-of-the-art models in image translation tasks.
  arXiv  Detail & Related papers  (2023-06-07T12:56:56Z)
• Real-World Image Variation by Aligning Diffusion Inversion Chain [53.772004619296794]
  A domain gap exists between generated images and real-world images, which poses a challenge in generating high-quality variations of real-world images. We propose a novel inference pipeline called Real-world Image Variation by ALignment (RIVAL) Our pipeline enhances the generation quality of image variations by aligning the image generation process to the source image's inversion chain.
  arXiv  Detail & Related papers  (2023-05-30T04:09:47Z)
• Variational Bayesian Framework for Advanced Image Generation with Domain-Related Variables [29.827191184889898]
  We present a unified Bayesian framework for advanced conditional generative problems. We propose a variational Bayesian image translation network (VBITN) that enables multiple image translation and editing tasks.
  arXiv  Detail & Related papers  (2023-05-23T09:47:23Z)
• Semantic Image Synthesis via Diffusion Models [159.4285444680301]
  Denoising Diffusion Probabilistic Models (DDPMs) have achieved remarkable success in various image generation tasks. Recent work on semantic image synthesis mainly follows the emphde facto Generative Adversarial Nets (GANs)
  arXiv  Detail & Related papers  (2022-06-30T18:31:51Z)
• Fusion Models for Improved Visual Captioning [18.016295296424413]
  This paper proposes a generic multimodal model fusion framework for caption generation and emendation. We employ the same fusion strategies to integrate a pretrained Masked Language Model (MLM) with a visual captioning model, viz. Show, Attend, and Tell. Our caption emendation experiments on three benchmark image captioning datasets, viz. Flickr8k, Flickr30k, and MSCOCO, show improvements over the baseline.
  arXiv  Detail & Related papers  (2020-10-28T21:55:25Z)
• Multimodal Image-to-Image Translation via Mutual Information Estimation and Maximization [16.54980086211836]
  Multimodal image-to-image translation (I2IT) aims to learn a conditional distribution that explores multiple possible images in the target domain given an input image in the source domain. Conditional generative adversarial networks (cGANs) are often adopted for modeling such a conditional distribution. We propose a method that explicitly estimates and maximizes the mutual information between the latent code and the output image in cGANs.
  arXiv  Detail & Related papers  (2020-08-08T14:09:23Z)
• MCMI: Multi-Cycle Image Translation with Mutual Information Constraints [40.556049046897115]
  We present a mutual information-based framework for unsupervised image-to-image translation. Our MCMI approach treats single-cycle image translation models as modules that can be used recurrently in a multi-cycle translation setting. We show that models trained with MCMI produce higher quality images and learn more semantically-relevant mappings.
  arXiv  Detail & Related papers  (2020-07-06T17:50:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.