SAM-helps-Shadow:When Segment Anything Model meet shadow removal

• URL: http://arxiv.org/abs/2306.06113v1
• Date: Thu, 1 Jun 2023 06:37:19 GMT
• Title: SAM-helps-Shadow:When Segment Anything Model meet shadow removal
• Authors: Xiaofeng Zhang, Chaochen Gu, Shanying Zhu
• Abstract summary: In this study, we innovatively adapted the SAM (Segment anything model) for shadow removal by introducing SAM-helps-Shadow. Our approach utilized the model's detection results as a potent prior for facilitating shadow detection, followed by shadow removal using a second-order deep unfolding network.
• Score: 8.643096072885909
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The challenges surrounding the application of image shadow removal to real-world images and not just constrained datasets like ISTD/SRD have highlighted an urgent need for zero-shot learning in this field. In this study, we innovatively adapted the SAM (Segment anything model) for shadow removal by introducing SAM-helps-Shadow, effectively integrating shadow detection and removal into a single stage. Our approach utilized the model's detection results as a potent prior for facilitating shadow detection, followed by shadow removal using a second-order deep unfolding network. The source code of SAM-helps-Shadow can be obtained from https://github.com/zhangbaijin/SAM-helps-Shadow.

Related papers

• SoftShadow: Leveraging Penumbra-Aware Soft Masks for Shadow Removal [35.16957947180504]
  We introduce novel soft shadow masks specifically designed for shadow removal. To achieve such soft masks, we propose a textitSoftShadow framework by leveraging the prior knowledge of pretrained SAM. This framework enables accurate predictions of penumbra (partially shaded regions) and umbra (fully shaded regions) areas while simultaneously facilitating end-to-end shadow removal.
  arXiv  Detail & Related papers  (2024-09-11T06:12:26Z)
• Shadow Removal Refinement via Material-Consistent Shadow Edges [33.8383848078524]
  On both sides of shadow edges traversing regions with the same material, the original color and textures should be the same if the shadow is removed properly. We fine-tune SAM, an image segmentation foundation model, to produce a shadow-invariant segmentation and then extract material-consistent shadow edges. We demonstrate the effectiveness of our method in improving shadow removal results on more challenging, in-the-wild images.
  arXiv  Detail & Related papers  (2024-09-10T20:16:28Z)
• AdapterShadow: Adapting Segment Anything Model for Shadow Detection [6.201928340999525]
  Segment anything model (SAM) has shown its spectacular performance in segmenting universal objects. However, it fails to segment specific targets, e.g., shadow images or lesions in medical images. We propose AdapterShadow, which adapts SAM model for shadow detection.
  arXiv  Detail & Related papers  (2023-11-15T11:51:10Z)
• Progressive Recurrent Network for Shadow Removal [99.1928825224358]
  Single-image shadow removal is a significant task that is still unresolved. Most existing deep learning-based approaches attempt to remove the shadow directly, which can not deal with the shadow well. We propose a simple but effective Progressive Recurrent Network (PRNet) to remove the shadow progressively.
  arXiv  Detail & Related papers  (2023-11-01T11:42:45Z)
• SILT: Shadow-aware Iterative Label Tuning for Learning to Detect Shadows from Noisy Labels [53.30604926018168]
  We propose SILT, the Shadow-aware Iterative Label Tuning framework, which explicitly considers noise in shadow labels and trains the deep model in a self-training manner. We also devise a simple yet effective label tuning strategy with global-local fusion and shadow-aware filtering to encourage the network to make significant refinements on the noisy labels. Our results show that even a simple U-Net trained with SILT can outperform all state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2023-08-23T11:16:36Z)
• Detect Any Shadow: Segment Anything for Video Shadow Detection [105.19693622157462]
  We propose ShadowSAM, a framework for fine-tuning segment anything model (SAM) to detect shadows. By combining it with long short-term attention mechanism, we extend its capability for efficient video shadow detection. Our method exhibits accelerated inference speed compared to previous video shadow detection approaches.
  arXiv  Detail & Related papers  (2023-05-26T07:39:10Z)
• DeS3: Adaptive Attention-driven Self and Soft Shadow Removal using ViT Similarity [54.831083157152136]
  We present a method that removes hard, soft and self shadows based on adaptive attention and ViT similarity. Our method outperforms state-of-the-art methods on the SRD, AISTD, LRSS, USR and UIUC datasets.
  arXiv  Detail & Related papers  (2022-11-15T12:15:29Z)
• R2D: Learning Shadow Removal to Enhance Fine-Context Shadow Detection [64.10636296274168]
  Current shadow detection methods perform poorly when detecting shadow regions that are small, unclear or have blurry edges. We propose a new method called Restore to Detect (R2D), where a deep neural network is trained for restoration (shadow removal) We show that our proposed method R2D improves the shadow detection performance while being able to detect fine context better compared to the other recent methods.
  arXiv  Detail & Related papers  (2021-09-20T15:09:22Z)
• From Shadow Generation to Shadow Removal [19.486543304598264]
  We propose a new G2R-ShadowNet which leverages shadow generation for weakly-supervised shadow removal. The proposed G2R-ShadowNet consists of three sub-networks for shadow generation, shadow removal and refinement. In particular, the shadow generation sub-net stylises non-shadow regions to be shadow ones, leading to paired data for training the shadow-removal sub-net.
  arXiv  Detail & Related papers  (2021-03-24T05:49:08Z)
• Physics-based Shadow Image Decomposition for Shadow Removal [36.41558227710456]
  We propose a novel deep learning method for shadow removal. Inspired by physical models of shadow formation, we use a linear illumination transformation to model the shadow effects in the image. We train and test our framework on the most challenging shadow removal dataset.
  arXiv  Detail & Related papers  (2020-12-23T23:06:38Z)
• Self-Supervised Shadow Removal [130.6657167667636]
  We propose an unsupervised single image shadow removal solution via self-supervised learning by using a conditioned mask. In contrast to existing literature, we do not require paired shadowed and shadow-free images, instead we rely on self-supervision and jointly learn deep models to remove and add shadows to images.
  arXiv  Detail & Related papers  (2020-10-22T11:33:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.