Fuzzy-Conditioned Diffusion and Diffusion Projection Attention Applied to Facial Image Correction

• URL: http://arxiv.org/abs/2306.14891v2
• Date: Sat, 1 Jul 2023 22:22:14 GMT
• Title: Fuzzy-Conditioned Diffusion and Diffusion Projection Attention Applied to Facial Image Correction
• Authors: Majed El Helou
• Abstract summary: We derive a fuzzy-conditioned diffusion, where implicit diffusion priors can be exploited with controllable strength. We propose an application to facial image correction, where we combine our fuzzy-conditioned diffusion with diffusion-derived attention maps.
• Score: 14.34815548338413
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Image diffusion has recently shown remarkable performance in image synthesis and implicitly as an image prior. Such a prior has been used with conditioning to solve the inpainting problem, but only supporting binary user-based conditioning. We derive a fuzzy-conditioned diffusion, where implicit diffusion priors can be exploited with controllable strength. Our fuzzy conditioning can be applied pixel-wise, enabling the modification of different image components to varying degrees. Additionally, we propose an application to facial image correction, where we combine our fuzzy-conditioned diffusion with diffusion-derived attention maps. Our map estimates the degree of anomaly, and we obtain it by projecting on the diffusion space. We show how our approach also leads to interpretable and autonomous facial image correction.

Related papers

• Reinforced Diffusion: Learning to Push the Limits of Anisotropic Diffusion for Image Denoising [57.226775716102765]
  We describe a trainable anisotropic diffusion framework based on reinforcement learning.<n>By modeling the denoising process as a series of naive diffusion actions with iterations order learned by deep Q-learning, we propose an effective diffusion-based image denoiser.
  arXiv  Detail & Related papers  (2025-12-30T07:23:15Z)
• Immiscible Diffusion: Accelerating Diffusion Training with Noise Assignment [56.609042046176555]
  suboptimal noise-data mapping leads to slow training of diffusion models. Drawing inspiration from the immiscibility phenomenon in physics, we propose Immiscible Diffusion. Our approach is remarkably simple, requiring only one line of code to restrict the diffuse-able area for each image.
  arXiv  Detail & Related papers  (2024-06-18T06:20:42Z)
• Smooth Diffusion: Crafting Smooth Latent Spaces in Diffusion Models [82.8261101680427]
  Smooth latent spaces ensure that a perturbation on an input latent corresponds to a steady change in the output image. This property proves beneficial in downstream tasks, including image inversion, inversion, and editing. We propose Smooth Diffusion, a new category of diffusion models that can be simultaneously high-performing and smooth.
  arXiv  Detail & Related papers  (2023-12-07T16:26:23Z)
• Prompt-tuning latent diffusion models for inverse problems [72.13952857287794]
  We propose a new method for solving imaging inverse problems using text-to-image latent diffusion models as general priors. Our method, called P2L, outperforms both image- and latent-diffusion model-based inverse problem solvers on a variety of tasks, such as super-resolution, deblurring, and inpainting.
  arXiv  Detail & Related papers  (2023-10-02T11:31:48Z)
• MaskDiffusion: Boosting Text-to-Image Consistency with Conditional Mask [84.84034179136458]
  A crucial factor leading to the text-image mismatch issue is the inadequate cross-modality relation learning. We propose an adaptive mask, which is conditioned on the attention maps and the prompt embeddings, to dynamically adjust the contribution of each text token to the image features. Our method, termed MaskDiffusion, is training-free and hot-pluggable for popular pre-trained diffusion models.
  arXiv  Detail & Related papers  (2023-09-08T15:53:37Z)
• Not All Steps are Created Equal: Selective Diffusion Distillation for Image Manipulation [23.39614544877529]
  Conditional diffusion models have demonstrated impressive performance in image manipulation tasks. Adding too much noise affects the fidelity of the image while adding too little affects its editability. We propose a novel framework, Diffusion Selective Distillation (SDD), that ensures both the fidelity and editability of images.
  arXiv  Detail & Related papers  (2023-07-17T12:42:56Z)
• Diffusion in Diffusion: Cyclic One-Way Diffusion for Text-Vision-Conditioned Generation [11.80682025950519]
  In this work, we investigate the diffusion (physics) in diffusion (machine learning) properties. We propose our Cyclic One-Way Diffusion (COW) method to control the direction of diffusion phenomenon. Our method provides a novel perspective to understand the task needs and is applicable to a wider range of customization scenarios.
  arXiv  Detail & Related papers  (2023-06-14T05:25:06Z)
• A Variational Perspective on Solving Inverse Problems with Diffusion Models [101.831766524264]
  Inverse tasks can be formulated as inferring a posterior distribution over data. This is however challenging in diffusion models since the nonlinear and iterative nature of the diffusion process renders the posterior intractable. We propose a variational approach that by design seeks to approximate the true posterior distribution.
  arXiv  Detail & Related papers  (2023-05-07T23:00:47Z)
• Diffusion Models Generate Images Like Painters: an Analytical Theory of Outline First, Details Later [1.8416014644193066]
  We observe that the reverse diffusion process that underlies image generation has the following properties. Individual trajectories tend to be low-dimensional and resemble 2D rotations' We find that this solution accurately describes the initial phase of image generation for pretrained models.
  arXiv  Detail & Related papers  (2023-03-04T20:08:57Z)
• ADIR: Adaptive Diffusion for Image Reconstruction [46.838084286784195]
  We propose a conditional sampling scheme that exploits the prior learned by diffusion models. We then combine it with a novel approach for adapting pretrained diffusion denoising networks to their input. We show that our proposed adaptive diffusion for image reconstruction' approach achieves a significant improvement in the super-resolution, deblurring, and text-based editing tasks.
  arXiv  Detail & Related papers  (2022-12-06T18:39:58Z)
• SinDiffusion: Learning a Diffusion Model from a Single Natural Image [159.4285444680301]
  We present SinDiffusion, leveraging denoising diffusion models to capture internal distribution of patches from a single natural image. It is based on two core designs. First, SinDiffusion is trained with a single model at a single scale instead of multiple models with progressive growing of scales. Second, we identify that a patch-level receptive field of the diffusion network is crucial and effective for capturing the image's patch statistics.
  arXiv  Detail & Related papers  (2022-11-22T18:00:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.