Latent Feature-Guided Diffusion Models for Shadow Removal

• URL: http://arxiv.org/abs/2312.02156v1
• Date: Mon, 4 Dec 2023 18:59:55 GMT
• Title: Latent Feature-Guided Diffusion Models for Shadow Removal
• Authors: Kangfu Mei and Luis Figueroa and Zhe Lin and Zhihong Ding and Scott Cohen and Vishal M. Patel
• Abstract summary: We propose the use of diffusion models as they offer a promising approach to gradually refine the details of shadow regions during the diffusion process. Our method improves this process by conditioning on a learned latent feature space that inherits the characteristics of shadow-free images. We demonstrate the effectiveness of our approach which outperforms the previous best method by 13% in terms of RMSE on the AISTD dataset.
• Score: 50.02857194218859
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recovering textures under shadows has remained a challenging problem due to the difficulty of inferring shadow-free scenes from shadow images. In this paper, we propose the use of diffusion models as they offer a promising approach to gradually refine the details of shadow regions during the diffusion process. Our method improves this process by conditioning on a learned latent feature space that inherits the characteristics of shadow-free images, thus avoiding the limitation of conventional methods that condition on degraded images only. Additionally, we propose to alleviate potential local optima during training by fusing noise features with the diffusion network. We demonstrate the effectiveness of our approach which outperforms the previous best method by 13% in terms of RMSE on the AISTD dataset. Further, we explore instance-level shadow removal, where our model outperforms the previous best method by 82% in terms of RMSE on the DESOBA dataset.

Related papers

• Generative Portrait Shadow Removal [27.98144439007323]
  We introduce a high-fidelity portrait shadow removal model that can effectively enhance the image of a portrait. Our method also demonstrates robustness to diverse subjects captured in real environments.
  arXiv  Detail & Related papers  (2024-10-07T22:09:22Z)
• Diff-Shadow: Global-guided Diffusion Model for Shadow Removal [46.41983327564438]
  We propose Diff-Shadow, a global-guided diffusion model for high-quality shadow removal. Our method achieves a significant improvement in terms of PSNR, increasing from 32.33dB to 33.69dB on the SRD dataset.
  arXiv  Detail & Related papers  (2024-07-23T06:42:55Z)
• Inhomogeneous illumination image enhancement under ex-tremely low visibility condition [3.534798835599242]
  Imaging through dense fog presents unique challenges, with essential visual information crucial for applications like object detection and recognition obscured, thereby hindering conventional image processing methods. We introduce in this paper a novel method that adaptively filters background illumination based on Structural Differential and Integral Filtering (F) to enhance only vital signal information. Our findings demonstrate that our proposed method significantly enhances signal clarity under extremely low visibility conditions and out-performs existing techniques, offering substantial improvements for deep fog imaging applications.
  arXiv  Detail & Related papers  (2024-04-26T16:09:42Z)
• ReNoise: Real Image Inversion Through Iterative Noising [62.96073631599749]
  We introduce an inversion method with a high quality-to-operation ratio, enhancing reconstruction accuracy without increasing the number of operations. We evaluate the performance of our ReNoise technique using various sampling algorithms and models, including recent accelerated diffusion models.
  arXiv  Detail & Related papers  (2024-03-21T17:52:08Z)
• Efficient Diffusion Model for Image Restoration by Residual Shifting [63.02725947015132]
  This study proposes a novel and efficient diffusion model for image restoration. Our method avoids the need for post-acceleration during inference, thereby avoiding the associated performance deterioration. Our method achieves superior or comparable performance to current state-of-the-art methods on three classical IR tasks.
  arXiv  Detail & Related papers  (2024-03-12T05:06:07Z)
• Deshadow-Anything: When Segment Anything Model Meets Zero-shot shadow removal [8.555176637147648]
  We develop Deshadow-Anything, considering the generalization of large-scale datasets, to achieve image shadow removal. The diffusion model can diffuse along the edges and textures of an image, helping to remove shadows while preserving the details of the image. Experiments on shadow removal tasks demonstrate that these methods can effectively improve image restoration performance.
  arXiv  Detail & Related papers  (2023-09-21T01:35:13Z)
• ExposureDiffusion: Learning to Expose for Low-light Image Enhancement [87.08496758469835]
  This work addresses the issue by seamlessly integrating a diffusion model with a physics-based exposure model. Our method obtains significantly improved performance and reduced inference time compared with vanilla diffusion models. The proposed framework can work with both real-paired datasets, SOTA noise models, and different backbone networks.
  arXiv  Detail & Related papers  (2023-07-15T04:48:35Z)
• Low-Light Image Enhancement with Wavelet-based Diffusion Models [50.632343822790006]
  Diffusion models have achieved promising results in image restoration tasks, yet suffer from time-consuming, excessive computational resource consumption, and unstable restoration. We propose a robust and efficient Diffusion-based Low-Light image enhancement approach, dubbed DiffLL.
  arXiv  Detail & Related papers  (2023-06-01T03:08:28Z)
• ShadowDiffusion: When Degradation Prior Meets Diffusion Model for Shadow Removal [74.86415440438051]
  We propose a unified diffusion framework that integrates both the image and degradation priors for highly effective shadow removal. Our model achieves a significant improvement in terms of PSNR, increasing from 31.69dB to 34.73dB over SRD dataset.
  arXiv  Detail & Related papers  (2022-12-09T07:48:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.