Towards Hard and Soft Shadow Removal via Dual-Branch Separation Network and Vision Transformer

• URL: http://arxiv.org/abs/2501.01864v2
• Date: Wed, 19 Feb 2025 07:30:17 GMT
• Title: Towards Hard and Soft Shadow Removal via Dual-Branch Separation Network and Vision Transformer
• Authors: Jiajia Liang,
• Abstract summary: We propose a dual-path model that processes hard and soft shadows separately. The model classifies shadow types and processes them through appropriate paths to produce shadow-free outputs. Our model outperforms state-of-the-art methods and achieves 2.905 RMSE value on the ISTD dataset.
• Score: 0.0
• License:
• Abstract: Image shadow removal is a crucial task in computer vision. In real-world scenes, shadows alter image color and brightness, posing challenges for perception and texture recognition. Traditional and deep learning methods often overlook the distinct needs for handling hard and soft shadows, thereby lacking detailed processing to specifically address each type of shadow in images.We propose a dual-path model that processes these shadows separately using specially designed loss functions to accomplish the hard and soft shadow removal. The model classifies shadow types and processes them through appropriate paths to produce shadow-free outputs, integrating a Vision Transformer with UNet++ for enhanced edge detail and feature fusion. Our model outperforms state-of-the-art methods and achieves 2.905 RMSE value on the ISTD dataset, which demonstrates greater effectiveness than typical single-path approaches.

Related papers

• MetaShadow: Object-Centered Shadow Detection, Removal, and Synthesis [64.00425120075045]
  Shadows are often under-considered or even ignored in image editing applications, limiting the realism of the edited results. In this paper, we introduce MetaShadow, a three-in-one versatile framework that enables detection, removal, and controllable synthesis of shadows in natural images in an object-centered fashion.
  arXiv  Detail & Related papers  (2024-12-03T18:04:42Z)
• Soft-Hard Attention U-Net Model and Benchmark Dataset for Multiscale Image Shadow Removal [2.999888908665659]
  This study proposes a novel deep learning architecture, named Soft-Hard Attention U-net (SHAU), focusing on multiscale shadow removal. It provides a novel synthetic dataset, named Multiscale Shadow Removal dataset (MSRD), containing complex shadow patterns of multiple scales. The results demonstrate the effectiveness of SHAU over the relevant state-of-the-art shadow removal methods across various benchmark datasets.
  arXiv  Detail & Related papers  (2024-08-07T12:42:06Z)
• Single-Image Shadow Removal Using Deep Learning: A Comprehensive Survey [78.84004293081631]
  The patterns of shadows are arbitrary, varied, and often have highly complex trace structures. The degradation caused by shadows is spatially non-uniform, resulting in inconsistencies in illumination and color between shadow and non-shadow areas. Recent developments in this field are primarily driven by deep learning-based solutions.
  arXiv  Detail & Related papers  (2024-07-11T20:58:38Z)
• Cross-Modal Spherical Aggregation for Weakly Supervised Remote Sensing Shadow Removal [22.4845448174729]
  We propose a weakly supervised shadow removal network with a spherical feature space, dubbed S2-ShadowNet, to explore the best of both worlds for visible and infrared modalities. Specifically, we employ a modal translation (visible-to-infrared) model to learn the cross-domain mapping, thus generating realistic infrared samples. We contribute a large-scale weakly supervised shadow removal benchmark, including 4000 shadow images with corresponding shadow masks.
  arXiv  Detail & Related papers  (2024-06-25T11:14:09Z)
• SDDNet: Style-guided Dual-layer Disentanglement Network for Shadow Detection [85.16141353762445]
  We treat the input shadow image as a composition of a background layer and a shadow layer, and design a Style-guided Dual-layer Disentanglement Network to model these layers independently. Our model effectively minimizes the detrimental effects of background color, yielding superior performance on three public datasets with a real-time inference speed of 32 FPS.
  arXiv  Detail & Related papers  (2023-08-17T12:10:51Z)
• Learning Restoration is Not Enough: Transfering Identical Mapping for Single-Image Shadow Removal [19.391619888009064]
  State-of-the-art shadow removal methods train deep neural networks on collected shadow & shadow-free image pairs. We find that two tasks exhibit poor compatibility, and using shared weights for these two tasks could lead to the model being optimized towards only one task. We propose to handle these two tasks separately and leverage the identical mapping results to guide the shadow restoration in an iterative manner.
  arXiv  Detail & Related papers  (2023-05-18T01:36:23Z)
• ShaDocNet: Learning Spatial-Aware Tokens in Transformer for Document Shadow Removal [53.01990632289937]
  We propose a Transformer-based model for document shadow removal. It uses shadow context encoding and decoding in both shadow and shadow-free regions.
  arXiv  Detail & Related papers  (2022-11-30T01:46:29Z)
• Shadow-Aware Dynamic Convolution for Shadow Removal [80.82708225269684]
  We introduce a novel Shadow-Aware Dynamic Convolution (SADC) module to decouple the interdependence between the shadow region and the non-shadow region. Inspired by the fact that the color mapping of the non-shadow region is easier to learn, our SADC processes the non-shadow region with a lightweight convolution module. We develop a novel intra-convolution distillation loss to strengthen the information flow from the non-shadow region to the shadow region.
  arXiv  Detail & Related papers  (2022-05-10T14:00:48Z)
• Self-Supervised Shadow Removal [130.6657167667636]
  We propose an unsupervised single image shadow removal solution via self-supervised learning by using a conditioned mask. In contrast to existing literature, we do not require paired shadowed and shadow-free images, instead we rely on self-supervision and jointly learn deep models to remove and add shadows to images.
  arXiv  Detail & Related papers  (2020-10-22T11:33:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.