ShadowRefiner: Towards Mask-free Shadow Removal via Fast Fourier Transformer

• URL: http://arxiv.org/abs/2406.02559v2
• Date: Wed, 3 Jul 2024 02:38:39 GMT
• Title: ShadowRefiner: Towards Mask-free Shadow Removal via Fast Fourier Transformer
• Authors: Wei Dong, Han Zhou, Yuqiong Tian, Jingke Sun, Xiaohong Liu, Guangtao Zhai, Jun Chen,
• Abstract summary: Shadow-affected images often exhibit pronounced spatial discrepancies in color and illumination. We introduce a mask-free Shadow Removal and Refinement network (ShadowRefiner) via Fast Fourier Transformer. Our method wins the championship in the Perceptual Track and achieves the second best performance in the Fidelity Track of NTIRE 2024 Image Shadow Removal Challenge.
• Score: 41.008740643546226
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Shadow-affected images often exhibit pronounced spatial discrepancies in color and illumination, consequently degrading various vision applications including object detection and segmentation systems. To effectively eliminate shadows in real-world images while preserving intricate details and producing visually compelling outcomes, we introduce a mask-free Shadow Removal and Refinement network (ShadowRefiner) via Fast Fourier Transformer. Specifically, the Shadow Removal module in our method aims to establish effective mappings between shadow-affected and shadow-free images via spatial and frequency representation learning. To mitigate the pixel misalignment and further improve the image quality, we propose a novel Fast-Fourier Attention based Transformer (FFAT) architecture, where an innovative attention mechanism is designed for meticulous refinement. Our method wins the championship in the Perceptual Track and achieves the second best performance in the Fidelity Track of NTIRE 2024 Image Shadow Removal Challenge. Besides, comprehensive experiment result also demonstrate the compelling effectiveness of our proposed method. The code is publicly available: https://github.com/movingforward100/Shadow_R.

Related papers

• Towards Hard and Soft Shadow Removal via Dual-Branch Separation Network and Vision Transformer [0.0]
  We propose a dual-path model that processes hard and soft shadows separately. The model classifies shadow types and processes them through appropriate paths to produce shadow-free outputs. Our model outperforms state-of-the-art methods and achieves 2.905 RMSE value on the ISTD dataset.
  arXiv  Detail & Related papers  (2025-01-03T15:29:46Z)
• MetaShadow: Object-Centered Shadow Detection, Removal, and Synthesis [64.00425120075045]
  Shadows are often under-considered or even ignored in image editing applications, limiting the realism of the edited results. In this paper, we introduce MetaShadow, a three-in-one versatile framework that enables detection, removal, and controllable synthesis of shadows in natural images in an object-centered fashion.
  arXiv  Detail & Related papers  (2024-12-03T18:04:42Z)
• Single-Image Shadow Removal Using Deep Learning: A Comprehensive Survey [78.84004293081631]
  The patterns of shadows are arbitrary, varied, and often have highly complex trace structures. The degradation caused by shadows is spatially non-uniform, resulting in inconsistencies in illumination and color between shadow and non-shadow areas. Recent developments in this field are primarily driven by deep learning-based solutions.
  arXiv  Detail & Related papers  (2024-07-11T20:58:38Z)
• Cross-Modal Spherical Aggregation for Weakly Supervised Remote Sensing Shadow Removal [22.4845448174729]
  We propose a weakly supervised shadow removal network with a spherical feature space, dubbed S2-ShadowNet, to explore the best of both worlds for visible and infrared modalities. Specifically, we employ a modal translation (visible-to-infrared) model to learn the cross-domain mapping, thus generating realistic infrared samples. We contribute a large-scale weakly supervised shadow removal benchmark, including 4000 shadow images with corresponding shadow masks.
  arXiv  Detail & Related papers  (2024-06-25T11:14:09Z)
• ShadowRemovalNet: Efficient Real-Time Shadow Removal [3.0516727053033392]
  ShadowRemovalNet is a novel method for real-time image processing on resource-constrained hardware. It achieves significantly higher frame rates compared to existing methods. It does not require a separate shadow mask during inference.
  arXiv  Detail & Related papers  (2024-03-13T00:04:07Z)
• Progressive Recurrent Network for Shadow Removal [99.1928825224358]
  Single-image shadow removal is a significant task that is still unresolved. Most existing deep learning-based approaches attempt to remove the shadow directly, which can not deal with the shadow well. We propose a simple but effective Progressive Recurrent Network (PRNet) to remove the shadow progressively.
  arXiv  Detail & Related papers  (2023-11-01T11:42:45Z)
• Leveraging Inpainting for Single-Image Shadow Removal [29.679542372017373]
  In this work, we find that pretraining shadow removal networks on the image inpainting dataset can reduce the shadow remnants significantly. A naive encoder-decoder network gets competitive restoration quality w.r.t. the state-of-the-art methods via only 10% shadow & shadow-free image pairs. Inspired by these observations, we formulate shadow removal as an adaptive fusion task that takes advantage of both shadow removal and image inpainting.
  arXiv  Detail & Related papers  (2023-02-10T16:21:07Z)
• ShaDocNet: Learning Spatial-Aware Tokens in Transformer for Document Shadow Removal [53.01990632289937]
  We propose a Transformer-based model for document shadow removal. It uses shadow context encoding and decoding in both shadow and shadow-free regions.
  arXiv  Detail & Related papers  (2022-11-30T01:46:29Z)
• Physics-based Shadow Image Decomposition for Shadow Removal [36.41558227710456]
  We propose a novel deep learning method for shadow removal. Inspired by physical models of shadow formation, we use a linear illumination transformation to model the shadow effects in the image. We train and test our framework on the most challenging shadow removal dataset.
  arXiv  Detail & Related papers  (2020-12-23T23:06:38Z)
• Self-Supervised Shadow Removal [130.6657167667636]
  We propose an unsupervised single image shadow removal solution via self-supervised learning by using a conditioned mask. In contrast to existing literature, we do not require paired shadowed and shadow-free images, instead we rely on self-supervision and jointly learn deep models to remove and add shadows to images.
  arXiv  Detail & Related papers  (2020-10-22T11:33:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.