CODE: Confident Ordinary Differential Editing

• URL: http://arxiv.org/abs/2408.12418v1
• Date: Thu, 22 Aug 2024 14:12:20 GMT
• Title: CODE: Confident Ordinary Differential Editing
• Authors: Bastien van Delft, Tommaso Martorella, Alexandre Alahi,
• Abstract summary: Confident Ordinary Differential Editing (CODE) is a novel approach for image synthesis that effectively handles Out-of-Distribution (OoD) guidance images. CODE enhances images through score-based updates along the probability-flow Ordinary Differential Equation (ODE) trajectory. Our method operates in a fully blind manner, relying solely on a pre-trained generative model.
• Score: 62.83365660727034
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Conditioning image generation facilitates seamless editing and the creation of photorealistic images. However, conditioning on noisy or Out-of-Distribution (OoD) images poses significant challenges, particularly in balancing fidelity to the input and realism of the output. We introduce Confident Ordinary Differential Editing (CODE), a novel approach for image synthesis that effectively handles OoD guidance images. Utilizing a diffusion model as a generative prior, CODE enhances images through score-based updates along the probability-flow Ordinary Differential Equation (ODE) trajectory. This method requires no task-specific training, no handcrafted modules, and no assumptions regarding the corruptions affecting the conditioning image. Our method is compatible with any diffusion model. Positioned at the intersection of conditional image generation and blind image restoration, CODE operates in a fully blind manner, relying solely on a pre-trained generative model. Our method introduces an alternative approach to blind restoration: instead of targeting a specific ground truth image based on assumptions about the underlying corruption, CODE aims to increase the likelihood of the input image while maintaining fidelity. This results in the most probable in-distribution image around the input. Our contributions are twofold. First, CODE introduces a novel editing method based on ODE, providing enhanced control, realism, and fidelity compared to its SDE-based counterpart. Second, we introduce a confidence interval-based clipping method, which improves CODE's effectiveness by allowing it to disregard certain pixels or information, thus enhancing the restoration process in a blind manner. Experimental results demonstrate CODE's effectiveness over existing methods, particularly in scenarios involving severe degradation or OoD inputs.

Related papers

• One-step Generative Diffusion for Realistic Extreme Image Rescaling [47.89362819768323]
  We propose a novel framework called One-Step Image Rescaling Diffusion (OSIRDiff) for extreme image rescaling. OSIRDiff performs rescaling operations in the latent space of a pre-trained autoencoder. It effectively leverages powerful natural image priors learned by a pre-trained text-to-image diffusion model.
  arXiv  Detail & Related papers  (2024-08-17T09:51:42Z)
• Coarse-to-Fine Latent Diffusion for Pose-Guided Person Image Synthesis [65.7968515029306]
  We propose a novel Coarse-to-Fine Latent Diffusion (CFLD) method for Pose-Guided Person Image Synthesis (PGPIS) A perception-refined decoder is designed to progressively refine a set of learnable queries and extract semantic understanding of person images as a coarse-grained prompt.
  arXiv  Detail & Related papers  (2024-02-28T06:07:07Z)
• Contrastive Denoising Score for Text-guided Latent Diffusion Image Editing [58.48890547818074]
  We present a powerful modification of Contrastive Denoising Score (CUT) for latent diffusion models (LDM) Our approach enables zero-shot imageto-image translation and neural field (NeRF) editing, achieving structural correspondence between the input and output.
  arXiv  Detail & Related papers  (2023-11-30T15:06:10Z)
• Unified Concept Editing in Diffusion Models [53.30378722979958]
  We present a method that tackles all issues with a single approach. Our method, Unified Concept Editing (UCE), edits the model without training using a closed-form solution. We demonstrate scalable simultaneous debiasing, style erasure, and content moderation by editing text-to-image projections.
  arXiv  Detail & Related papers  (2023-08-25T17:59:59Z)
• Eliminating Contextual Prior Bias for Semantic Image Editing via Dual-Cycle Diffusion [35.95513392917737]
  A novel approach called Dual-Cycle Diffusion generates an unbiased mask to guide image editing. Our experiments demonstrate the effectiveness of the proposed method, as it significantly improves the D-CLIP score from 0.272 to 0.283.
  arXiv  Detail & Related papers  (2023-02-05T14:30:22Z)
• Image Restoration with Mean-Reverting Stochastic Differential Equations [9.245782611878752]
  This paper presents a differential equation (SDE) approach for general-purpose image restoration. By simulating the corresponding reverse-time SDE, we are able to restore the origin of the low-quality image. Experiments show that our proposed method achieves highly competitive performance in quantitative comparisons on image deraining, deblurring, and denoising.
  arXiv  Detail & Related papers  (2023-01-27T13:20:48Z)
• SDEdit: Image Synthesis and Editing with Stochastic Differential Equations [113.35735935347465]
  We introduce Differential Editing (SDEdit), based on a recent generative model using differential equations (SDEs) Given an input image with user edits, we first add noise to the input according to an SDE, and subsequently denoise it by simulating the reverse SDE to gradually increase its likelihood under the prior. Our method does not require task-specific loss function designs, which are critical components for recent image editing methods based on GAN inversions.
  arXiv  Detail & Related papers  (2021-08-02T17:59:47Z)
• High Resolution Face Editing with Masked GAN Latent Code Optimization [0.0]
  Face editing is a popular research topic in the computer vision community. Recent proposed methods are based on either training a conditional encoder-decoder Generative Adversarial Network (GAN) in an end-to-end fashion or on defining an operation in the latent space of a pre-trained vanilla GAN generator model. We propose a GAN embedding optimization procedure with spatial and semantic constraints.
  arXiv  Detail & Related papers  (2021-03-20T08:39:41Z)
• Deep Variational Network Toward Blind Image Restoration [60.45350399661175]
  Blind image restoration is a common yet challenging problem in computer vision. We propose a novel blind image restoration method, aiming to integrate both the advantages of them. Experiments on two typical blind IR tasks, namely image denoising and super-resolution, demonstrate that the proposed method achieves superior performance over current state-of-the-arts.
  arXiv  Detail & Related papers  (2020-08-25T03:30:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.