OmniSR: Shadow Removal under Direct and Indirect Lighting

• URL: http://arxiv.org/abs/2410.01719v1
• Date: Wed, 2 Oct 2024 16:30:10 GMT
• Title: OmniSR: Shadow Removal under Direct and Indirect Lighting
• Authors: Jiamin Xu, Zelong Li, Yuxin Zheng, Chenyu Huang, Renshu Gu, Weiwei Xu, Gang Xu,
• Abstract summary: A significant challenge in removing shadows from indirect illumination is obtaining shadow-free images to train the shadow removal network. We propose a novel rendering pipeline for generating shadowed and shadow-free images under direct and indirect illumination. We also propose an innovative shadow removal network that explicitly integrates semantic and geometric priors through concatenation and attention mechanisms.
• Score: 16.90413085184936
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Shadows can originate from occlusions in both direct and indirect illumination. Although most current shadow removal research focuses on shadows caused by direct illumination, shadows from indirect illumination are often just as pervasive, particularly in indoor scenes. A significant challenge in removing shadows from indirect illumination is obtaining shadow-free images to train the shadow removal network. To overcome this challenge, we propose a novel rendering pipeline for generating shadowed and shadow-free images under direct and indirect illumination, and create a comprehensive synthetic dataset that contains over 30,000 image pairs, covering various object types and lighting conditions. We also propose an innovative shadow removal network that explicitly integrates semantic and geometric priors through concatenation and attention mechanisms. The experiments show that our method outperforms state-of-the-art shadow removal techniques and can effectively generalize to indoor and outdoor scenes under various lighting conditions, enhancing the overall effectiveness and applicability of shadow removal methods.

Related papers

• Shadow Removal Refinement via Material-Consistent Shadow Edges [33.8383848078524]
  On both sides of shadow edges traversing regions with the same material, the original color and textures should be the same if the shadow is removed properly. We fine-tune SAM, an image segmentation foundation model, to produce a shadow-invariant segmentation and then extract material-consistent shadow edges. We demonstrate the effectiveness of our method in improving shadow removal results on more challenging, in-the-wild images.
  arXiv  Detail & Related papers  (2024-09-10T20:16:28Z)
• Single-Image Shadow Removal Using Deep Learning: A Comprehensive Survey [78.84004293081631]
  The patterns of shadows are arbitrary, varied, and often have highly complex trace structures. The degradation caused by shadows is spatially non-uniform, resulting in inconsistencies in illumination and color between shadow and non-shadow areas. Recent developments in this field are primarily driven by deep learning-based solutions.
  arXiv  Detail & Related papers  (2024-07-11T20:58:38Z)
• Progressive Recurrent Network for Shadow Removal [99.1928825224358]
  Single-image shadow removal is a significant task that is still unresolved. Most existing deep learning-based approaches attempt to remove the shadow directly, which can not deal with the shadow well. We propose a simple but effective Progressive Recurrent Network (PRNet) to remove the shadow progressively.
  arXiv  Detail & Related papers  (2023-11-01T11:42:45Z)
• SIRe-IR: Inverse Rendering for BRDF Reconstruction with Shadow and Illumination Removal in High-Illuminance Scenes [51.50157919750782]
  We present SIRe-IR, an implicit neural rendering inverse approach that decomposes the scene into environment map, albedo, and roughness. By accurately modeling the indirect radiance field, normal, visibility, and direct light simultaneously, we are able to remove both shadows and indirect illumination. Even in the presence of intense illumination, our method recovers high-quality albedo and roughness with no shadow interference.
  arXiv  Detail & Related papers  (2023-10-19T10:44:23Z)
• ShadowFormer: Global Context Helps Image Shadow Removal [41.742799378751364]
  It is still challenging for the deep shadow removal model to exploit the global contextual correlation between shadow and non-shadow regions. We first propose a Retinex-based shadow model, from which we derive a novel transformer-based network, dubbed ShandowFormer. A multi-scale channel attention framework is employed to hierarchically capture the global information. We propose a Shadow-Interaction Module (SIM) with Shadow-Interaction Attention (SIA) in the bottleneck stage to effectively model the context correlation between shadow and non-shadow regions.
  arXiv  Detail & Related papers  (2023-02-03T10:54:52Z)
• Estimating Reflectance Layer from A Single Image: Integrating Reflectance Guidance and Shadow/Specular Aware Learning [66.36104525390316]
  We propose a two-stage learning method, including reflectance guidance and a Shadow/Specular-Aware (S-Aware) network to tackle the problem. In the first stage, an initial reflectance layer free from shadows and specularities is obtained with the constraint of novel losses. To further enforce the reflectance layer to be independent of shadows and specularities in the second-stage refinement, we introduce an S-Aware network that distinguishes the reflectance image from the input image.
  arXiv  Detail & Related papers  (2022-11-27T07:26:41Z)
• DeS3: Adaptive Attention-driven Self and Soft Shadow Removal using ViT Similarity [54.831083157152136]
  We present a method that removes hard, soft and self shadows based on adaptive attention and ViT similarity. Our method outperforms state-of-the-art methods on the SRD, AISTD, LRSS, USR and UIUC datasets.
  arXiv  Detail & Related papers  (2022-11-15T12:15:29Z)
• UnShadowNet: Illumination Critic Guided Contrastive Learning For Shadow Removal [14.898039056038789]
  We introduce a novel weakly supervised shadow removal framework UnShadowNet. It is composed of a DeShadower network responsible for the removal of the extracted shadow under the guidance of an Illumination network. We show that UnShadowNet can be easily extended to a fully-supervised set-up to exploit the ground-truth when available.
  arXiv  Detail & Related papers  (2022-03-29T11:17:02Z)
• Physics-based Shadow Image Decomposition for Shadow Removal [36.41558227710456]
  We propose a novel deep learning method for shadow removal. Inspired by physical models of shadow formation, we use a linear illumination transformation to model the shadow effects in the image. We train and test our framework on the most challenging shadow removal dataset.
  arXiv  Detail & Related papers  (2020-12-23T23:06:38Z)
• Self-Supervised Shadow Removal [130.6657167667636]
  We propose an unsupervised single image shadow removal solution via self-supervised learning by using a conditioned mask. In contrast to existing literature, we do not require paired shadowed and shadow-free images, instead we rely on self-supervision and jointly learn deep models to remove and add shadows to images.
  arXiv  Detail & Related papers  (2020-10-22T11:33:41Z)
• From Shadow Segmentation to Shadow Removal [34.762493656937366]
  The requirement for paired shadow and shadow-free images limits the size and diversity of shadow removal datasets. We propose a shadow removal method that can be trained using only shadow and non-shadow patches cropped from the shadow images themselves.
  arXiv  Detail & Related papers  (2020-08-01T14:00:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.