SpA-Former: Transformer image shadow detection and removal via spatial attention

• URL: http://arxiv.org/abs/2206.10910v1
• Date: Wed, 22 Jun 2022 08:30:22 GMT
• Title: SpA-Former: Transformer image shadow detection and removal via spatial attention
• Authors: Xiao Feng Zhang and Chao Chen Gu and Shan Ying Zhu
• Abstract summary: We propose an end-to-end SpA-Former to recover a shadow-free image from a single shaded image. Unlike traditional methods that require two steps for shadow detection and then shadow removal, the SpA-Former unifies these steps into one.
• Score: 8.643096072885909
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we propose an end-to-end SpA-Former to recover a shadow-free image from a single shaded image. Unlike traditional methods that require two steps for shadow detection and then shadow removal, the SpA-Former unifies these steps into one, which is a one-stage network capable of directly learning the mapping function between shadows and no shadows, it does not require a separate shadow detection. Thus, SpA-former is adaptable to real image de-shadowing for shadows projected on different semantic regions. SpA-Former consists of transformer layer and a series of joint Fourier transform residual blocks and two-wheel joint spatial attention. The network in this paper is able to handle the task while achieving a very fast processing efficiency. Our code is relased on https://github.com/ zhangbaijin/Spatial-Transformer-shadow-removal

Related papers

• SwinShadow: Shifted Window for Ambiguous Adjacent Shadow Detection [90.4751446041017]
  We present SwinShadow, a transformer-based architecture that fully utilizes the powerful shifted window mechanism for detecting adjacent shadows. The whole process can be divided into three parts: encoder, decoder, and feature integration. Experiments on three shadow detection benchmark datasets, SBU, UCF, and ISTD, demonstrate that our network achieves good performance in terms of balance error rate (BER)
  arXiv  Detail & Related papers  (2024-08-07T03:16:33Z)
• Progressive Recurrent Network for Shadow Removal [99.1928825224358]
  Single-image shadow removal is a significant task that is still unresolved. Most existing deep learning-based approaches attempt to remove the shadow directly, which can not deal with the shadow well. We propose a simple but effective Progressive Recurrent Network (PRNet) to remove the shadow progressively.
  arXiv  Detail & Related papers  (2023-11-01T11:42:45Z)
• Learning Restoration is Not Enough: Transfering Identical Mapping for Single-Image Shadow Removal [19.391619888009064]
  State-of-the-art shadow removal methods train deep neural networks on collected shadow & shadow-free image pairs. We find that two tasks exhibit poor compatibility, and using shared weights for these two tasks could lead to the model being optimized towards only one task. We propose to handle these two tasks separately and leverage the identical mapping results to guide the shadow restoration in an iterative manner.
  arXiv  Detail & Related papers  (2023-05-18T01:36:23Z)
• Leveraging Inpainting for Single-Image Shadow Removal [29.679542372017373]
  In this work, we find that pretraining shadow removal networks on the image inpainting dataset can reduce the shadow remnants significantly. A naive encoder-decoder network gets competitive restoration quality w.r.t. the state-of-the-art methods via only 10% shadow & shadow-free image pairs. Inspired by these observations, we formulate shadow removal as an adaptive fusion task that takes advantage of both shadow removal and image inpainting.
  arXiv  Detail & Related papers  (2023-02-10T16:21:07Z)
• ShadowFormer: Global Context Helps Image Shadow Removal [41.742799378751364]
  It is still challenging for the deep shadow removal model to exploit the global contextual correlation between shadow and non-shadow regions. We first propose a Retinex-based shadow model, from which we derive a novel transformer-based network, dubbed ShandowFormer. A multi-scale channel attention framework is employed to hierarchically capture the global information. We propose a Shadow-Interaction Module (SIM) with Shadow-Interaction Attention (SIA) in the bottleneck stage to effectively model the context correlation between shadow and non-shadow regions.
  arXiv  Detail & Related papers  (2023-02-03T10:54:52Z)
• Shadow Removal by High-Quality Shadow Synthesis [78.56549207362863]
  HQSS employs a shadow feature encoder and a generator to synthesize pseudo images. HQSS is observed to outperform the state-of-the-art methods on ISTD dataset, Video Shadow Removal dataset, and SRD dataset.
  arXiv  Detail & Related papers  (2022-12-08T06:52:52Z)
• DeS3: Adaptive Attention-driven Self and Soft Shadow Removal using ViT Similarity [54.831083157152136]
  We present a method that removes hard, soft and self shadows based on adaptive attention and ViT similarity. Our method outperforms state-of-the-art methods on the SRD, AISTD, LRSS, USR and UIUC datasets.
  arXiv  Detail & Related papers  (2022-11-15T12:15:29Z)
• CRFormer: A Cross-Region Transformer for Shadow Removal [27.67680052355886]
  We propose a novel cross-region transformer, namely CRFormer, for shadow removal. This is achieved by a carefully designed region-aware cross-attention operation. Experiments on ISTD, AISTD, SRD, and Video Shadow Removal datasets demonstrate the superiority of our method.
  arXiv  Detail & Related papers  (2022-07-04T17:33:02Z)
• Shadow-Aware Dynamic Convolution for Shadow Removal [80.82708225269684]
  We introduce a novel Shadow-Aware Dynamic Convolution (SADC) module to decouple the interdependence between the shadow region and the non-shadow region. Inspired by the fact that the color mapping of the non-shadow region is easier to learn, our SADC processes the non-shadow region with a lightweight convolution module. We develop a novel intra-convolution distillation loss to strengthen the information flow from the non-shadow region to the shadow region.
  arXiv  Detail & Related papers  (2022-05-10T14:00:48Z)
• R2D: Learning Shadow Removal to Enhance Fine-Context Shadow Detection [64.10636296274168]
  Current shadow detection methods perform poorly when detecting shadow regions that are small, unclear or have blurry edges. We propose a new method called Restore to Detect (R2D), where a deep neural network is trained for restoration (shadow removal) We show that our proposed method R2D improves the shadow detection performance while being able to detect fine context better compared to the other recent methods.
  arXiv  Detail & Related papers  (2021-09-20T15:09:22Z)
• Auto-Exposure Fusion for Single-Image Shadow Removal [23.178329688546032]
  Shadow removal is still a challenging task due to its inherent background-dependent and spatial-variant properties. Even powerful state-of-the-art deep neural networks could hardly recover traceless shadow-removed background. This paper proposes a new solution by formulating it as an exposure fusion problem to address the challenges.
  arXiv  Detail & Related papers  (2021-03-01T19:09:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.