ShadowFormer: Global Context Helps Image Shadow Removal

• URL: http://arxiv.org/abs/2302.01650v1
• Date: Fri, 3 Feb 2023 10:54:52 GMT
• Title: ShadowFormer: Global Context Helps Image Shadow Removal
• Authors: Lanqing Guo, Siyu Huang, Ding Liu, Hao Cheng, Bihan Wen
• Abstract summary: It is still challenging for the deep shadow removal model to exploit the global contextual correlation between shadow and non-shadow regions. We first propose a Retinex-based shadow model, from which we derive a novel transformer-based network, dubbed ShandowFormer. A multi-scale channel attention framework is employed to hierarchically capture the global information. We propose a Shadow-Interaction Module (SIM) with Shadow-Interaction Attention (SIA) in the bottleneck stage to effectively model the context correlation between shadow and non-shadow regions.
• Score: 41.742799378751364
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent deep learning methods have achieved promising results in image shadow removal. However, most of the existing approaches focus on working locally within shadow and non-shadow regions, resulting in severe artifacts around the shadow boundaries as well as inconsistent illumination between shadow and non-shadow regions. It is still challenging for the deep shadow removal model to exploit the global contextual correlation between shadow and non-shadow regions. In this work, we first propose a Retinex-based shadow model, from which we derive a novel transformer-based network, dubbed ShandowFormer, to exploit non-shadow regions to help shadow region restoration. A multi-scale channel attention framework is employed to hierarchically capture the global information. Based on that, we propose a Shadow-Interaction Module (SIM) with Shadow-Interaction Attention (SIA) in the bottleneck stage to effectively model the context correlation between shadow and non-shadow regions. We conduct extensive experiments on three popular public datasets, including ISTD, ISTD+, and SRD, to evaluate the proposed method. Our method achieves state-of-the-art performance by using up to 150X fewer model parameters.

Related papers

• ShadowMamba: State-Space Model with Boundary-Region Selective Scan for Shadow Removal [3.5734732877967392]
  This paper proposes a novel selective scanning method called boundary-region selective scanning. We name our model ShadowMamba, the first Mamba-based model for shadow removal.
  arXiv  Detail & Related papers  (2024-11-05T16:59:06Z)
• Single-Image Shadow Removal Using Deep Learning: A Comprehensive Survey [78.84004293081631]
  The patterns of shadows are arbitrary, varied, and often have highly complex trace structures. The degradation caused by shadows is spatially non-uniform, resulting in inconsistencies in illumination and color between shadow and non-shadow areas. Recent developments in this field are primarily driven by deep learning-based solutions.
  arXiv  Detail & Related papers  (2024-07-11T20:58:38Z)
• Delving into Dark Regions for Robust Shadow Detection [47.60700654394781]
  State-of-the-art deep methods tend to have higher error rates in differentiating shadow pixels from non-shadow pixels in dark regions. We propose a novel shadow detection approach that first learns global contextual cues over the entire image and then zooms into the dark regions to learn local shadow representations.
  arXiv  Detail & Related papers  (2024-02-21T09:07:07Z)
• Learning Restoration is Not Enough: Transfering Identical Mapping for Single-Image Shadow Removal [19.391619888009064]
  State-of-the-art shadow removal methods train deep neural networks on collected shadow & shadow-free image pairs. We find that two tasks exhibit poor compatibility, and using shared weights for these two tasks could lead to the model being optimized towards only one task. We propose to handle these two tasks separately and leverage the identical mapping results to guide the shadow restoration in an iterative manner.
  arXiv  Detail & Related papers  (2023-05-18T01:36:23Z)
• Shadow Removal by High-Quality Shadow Synthesis [78.56549207362863]
  HQSS employs a shadow feature encoder and a generator to synthesize pseudo images. HQSS is observed to outperform the state-of-the-art methods on ISTD dataset, Video Shadow Removal dataset, and SRD dataset.
  arXiv  Detail & Related papers  (2022-12-08T06:52:52Z)
• ShaDocNet: Learning Spatial-Aware Tokens in Transformer for Document Shadow Removal [53.01990632289937]
  We propose a Transformer-based model for document shadow removal. It uses shadow context encoding and decoding in both shadow and shadow-free regions.
  arXiv  Detail & Related papers  (2022-11-30T01:46:29Z)
• CNSNet: A Cleanness-Navigated-Shadow Network for Shadow Removal [4.951051823391577]
  We propose a shadow-oriented adaptive normalization (SOAN) module and a shadow-aware aggregation with transformer (SAAT) module based on the shadow mask. Under the guidance of the shadow mask, the SOAN module formulates the statistics from the non-shadow region and adaptively applies them to the shadow region for region-wise restoration. The SAAT module utilizes the shadow mask to precisely guide the restoration of each shadowed pixel by considering the highly relevant pixels from the shadow-free regions for global pixel-wise restoration.
  arXiv  Detail & Related papers  (2022-09-06T01:33:38Z)
• CRFormer: A Cross-Region Transformer for Shadow Removal [27.67680052355886]
  We propose a novel cross-region transformer, namely CRFormer, for shadow removal. This is achieved by a carefully designed region-aware cross-attention operation. Experiments on ISTD, AISTD, SRD, and Video Shadow Removal datasets demonstrate the superiority of our method.
  arXiv  Detail & Related papers  (2022-07-04T17:33:02Z)
• Shadow-Aware Dynamic Convolution for Shadow Removal [80.82708225269684]
  We introduce a novel Shadow-Aware Dynamic Convolution (SADC) module to decouple the interdependence between the shadow region and the non-shadow region. Inspired by the fact that the color mapping of the non-shadow region is easier to learn, our SADC processes the non-shadow region with a lightweight convolution module. We develop a novel intra-convolution distillation loss to strengthen the information flow from the non-shadow region to the shadow region.
  arXiv  Detail & Related papers  (2022-05-10T14:00:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.