ShadowMamba: State-Space Model with Boundary-Region Selective Scan for Shadow Removal

• URL: http://arxiv.org/abs/2411.03260v2
• Date: Sat, 08 Mar 2025 03:12:27 GMT
• Title: ShadowMamba: State-Space Model with Boundary-Region Selective Scan for Shadow Removal
• Authors: Xiujin Zhu, Chee-Onn Chow, Joon Huang Chuah,
• Abstract summary: Shadows cause sudden brightness changes in some areas, which can affect the accuracy of downstream tasks.<n>We propose a new boundary-region selective scanning mechanism that scans shadow, boundary, and non-shadow regions separately.<n>We design the first Mamba-based lightweight shadow removal model, called ShadowMamba.
• Score: 3.5734732877967392
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image shadow removal is a common low-level vision problem. Shadows cause sudden brightness changes in some areas, which can affect the accuracy of downstream tasks. Currently, Transformer-based shadow removal methods improve computational efficiency by using a window mechanism. However, this approach reduces the effective receptive field and weakens the ability to model long-range dependencies in shadow images. Recently, Mamba has achieved significant success in computer vision by modeling long-sequence information globally with linear complexity. However, when applied to shadow removal, its original scanning mechanism overlooks the semantic continuity along shadow boundaries, and the coherence within each region. To solve this issue, we propose a new boundary-region selective scanning mechanism that scans shadow, boundary, and non-shadow regions separately, making pixels of the same type closer in the sequence. This increases semantic continuity and helps the model understand local details better. Incorporating this idea, we design the first Mamba-based lightweight shadow removal model, called ShadowMamba. It uses a hierarchical combination U-Net structure, which effectively reduces the number of parameters and computational complexity. Shallow layers rely on our boundary-region selective scanning to capture local details, while deeper layers use global cross-scanning to learn global brightness features. Extensive experiments show that ShadowMamba outperforms current state-of-the-art models on ISTD+, ISTD, and SRD datasets, and it also requires fewer parameters and less computational cost. (Code will be made available upon paper acceptance.)

Related papers

• Shadow Removal Refinement via Material-Consistent Shadow Edges [33.8383848078524]
  On both sides of shadow edges traversing regions with the same material, the original color and textures should be the same if the shadow is removed properly. We fine-tune SAM, an image segmentation foundation model, to produce a shadow-invariant segmentation and then extract material-consistent shadow edges. We demonstrate the effectiveness of our method in improving shadow removal results on more challenging, in-the-wild images.
  arXiv  Detail & Related papers  (2024-09-10T20:16:28Z)
• Soft-Hard Attention U-Net Model and Benchmark Dataset for Multiscale Image Shadow Removal [2.999888908665659]
  This study proposes a novel deep learning architecture, named Soft-Hard Attention U-net (SHAU), focusing on multiscale shadow removal. It provides a novel synthetic dataset, named Multiscale Shadow Removal dataset (MSRD), containing complex shadow patterns of multiple scales. The results demonstrate the effectiveness of SHAU over the relevant state-of-the-art shadow removal methods across various benchmark datasets.
  arXiv  Detail & Related papers  (2024-08-07T12:42:06Z)
• SwinShadow: Shifted Window for Ambiguous Adjacent Shadow Detection [90.4751446041017]
  We present SwinShadow, a transformer-based architecture that fully utilizes the powerful shifted window mechanism for detecting adjacent shadows. The whole process can be divided into three parts: encoder, decoder, and feature integration. Experiments on three shadow detection benchmark datasets, SBU, UCF, and ISTD, demonstrate that our network achieves good performance in terms of balance error rate (BER)
  arXiv  Detail & Related papers  (2024-08-07T03:16:33Z)
• Single-Image Shadow Removal Using Deep Learning: A Comprehensive Survey [78.84004293081631]
  The patterns of shadows are arbitrary, varied, and often have highly complex trace structures. The degradation caused by shadows is spatially non-uniform, resulting in inconsistencies in illumination and color between shadow and non-shadow areas. Recent developments in this field are primarily driven by deep learning-based solutions.
  arXiv  Detail & Related papers  (2024-07-11T20:58:38Z)
• Delving into Dark Regions for Robust Shadow Detection [47.60700654394781]
  State-of-the-art deep methods tend to have higher error rates in differentiating shadow pixels from non-shadow pixels in dark regions. We propose a novel shadow detection approach that first learns global contextual cues over the entire image and then zooms into the dark regions to learn local shadow representations.
  arXiv  Detail & Related papers  (2024-02-21T09:07:07Z)
• Progressive Recurrent Network for Shadow Removal [99.1928825224358]
  Single-image shadow removal is a significant task that is still unresolved. Most existing deep learning-based approaches attempt to remove the shadow directly, which can not deal with the shadow well. We propose a simple but effective Progressive Recurrent Network (PRNet) to remove the shadow progressively.
  arXiv  Detail & Related papers  (2023-11-01T11:42:45Z)
• Learning Restoration is Not Enough: Transfering Identical Mapping for Single-Image Shadow Removal [19.391619888009064]
  State-of-the-art shadow removal methods train deep neural networks on collected shadow & shadow-free image pairs. We find that two tasks exhibit poor compatibility, and using shared weights for these two tasks could lead to the model being optimized towards only one task. We propose to handle these two tasks separately and leverage the identical mapping results to guide the shadow restoration in an iterative manner.
  arXiv  Detail & Related papers  (2023-05-18T01:36:23Z)
• ShadowFormer: Global Context Helps Image Shadow Removal [41.742799378751364]
  It is still challenging for the deep shadow removal model to exploit the global contextual correlation between shadow and non-shadow regions. We first propose a Retinex-based shadow model, from which we derive a novel transformer-based network, dubbed ShandowFormer. A multi-scale channel attention framework is employed to hierarchically capture the global information. We propose a Shadow-Interaction Module (SIM) with Shadow-Interaction Attention (SIA) in the bottleneck stage to effectively model the context correlation between shadow and non-shadow regions.
  arXiv  Detail & Related papers  (2023-02-03T10:54:52Z)
• Shadow Removal by High-Quality Shadow Synthesis [78.56549207362863]
  HQSS employs a shadow feature encoder and a generator to synthesize pseudo images. HQSS is observed to outperform the state-of-the-art methods on ISTD dataset, Video Shadow Removal dataset, and SRD dataset.
  arXiv  Detail & Related papers  (2022-12-08T06:52:52Z)
• ShaDocNet: Learning Spatial-Aware Tokens in Transformer for Document Shadow Removal [53.01990632289937]
  We propose a Transformer-based model for document shadow removal. It uses shadow context encoding and decoding in both shadow and shadow-free regions.
  arXiv  Detail & Related papers  (2022-11-30T01:46:29Z)
• CNSNet: A Cleanness-Navigated-Shadow Network for Shadow Removal [4.951051823391577]
  We propose a shadow-oriented adaptive normalization (SOAN) module and a shadow-aware aggregation with transformer (SAAT) module based on the shadow mask. Under the guidance of the shadow mask, the SOAN module formulates the statistics from the non-shadow region and adaptively applies them to the shadow region for region-wise restoration. The SAAT module utilizes the shadow mask to precisely guide the restoration of each shadowed pixel by considering the highly relevant pixels from the shadow-free regions for global pixel-wise restoration.
  arXiv  Detail & Related papers  (2022-09-06T01:33:38Z)
• LAB-Net: LAB Color-Space Oriented Lightweight Network for Shadow Removal [82.15476792337529]
  We present a novel lightweight deep neural network that processes shadow images in the LAB color space. The proposed network termed "LAB-Net", is motivated by the following three observations. Experimental results show that our LAB-Net well outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-08-27T15:34:15Z)
• Shadow-Aware Dynamic Convolution for Shadow Removal [80.82708225269684]
  We introduce a novel Shadow-Aware Dynamic Convolution (SADC) module to decouple the interdependence between the shadow region and the non-shadow region. Inspired by the fact that the color mapping of the non-shadow region is easier to learn, our SADC processes the non-shadow region with a lightweight convolution module. We develop a novel intra-convolution distillation loss to strengthen the information flow from the non-shadow region to the shadow region.
  arXiv  Detail & Related papers  (2022-05-10T14:00:48Z)
• R2D: Learning Shadow Removal to Enhance Fine-Context Shadow Detection [64.10636296274168]
  Current shadow detection methods perform poorly when detecting shadow regions that are small, unclear or have blurry edges. We propose a new method called Restore to Detect (R2D), where a deep neural network is trained for restoration (shadow removal) We show that our proposed method R2D improves the shadow detection performance while being able to detect fine context better compared to the other recent methods.
  arXiv  Detail & Related papers  (2021-09-20T15:09:22Z)
• Self-Supervised Shadow Removal [130.6657167667636]
  We propose an unsupervised single image shadow removal solution via self-supervised learning by using a conditioned mask. In contrast to existing literature, we do not require paired shadowed and shadow-free images, instead we rely on self-supervision and jointly learn deep models to remove and add shadows to images.
  arXiv  Detail & Related papers  (2020-10-22T11:33:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.