Robust Reflection Removal with Flash-only Cues in the Wild

• URL: http://arxiv.org/abs/2211.02914v2
• Date: Mon, 13 Nov 2023 23:16:13 GMT
• Title: Robust Reflection Removal with Flash-only Cues in the Wild
• Authors: Chenyang Lei, Xudong Jiang, Qifeng Chen
• Abstract summary: We propose a reflection-free cue for robust reflection removal from a pair of flash and ambient (no-flash) images. Our model outperforms state-of-the-art reflection removal approaches by more than 5.23dB in PSNR. We extend our approach to handheld photography to address the misalignment between the flash and no-flash pair.
• Score: 88.13531903652526
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a simple yet effective reflection-free cue for robust reflection removal from a pair of flash and ambient (no-flash) images. The reflection-free cue exploits a flash-only image obtained by subtracting the ambient image from the corresponding flash image in raw data space. The flash-only image is equivalent to an image taken in a dark environment with only a flash on. This flash-only image is visually reflection-free and thus can provide robust cues to infer the reflection in the ambient image. Since the flash-only image usually has artifacts, we further propose a dedicated model that not only utilizes the reflection-free cue but also avoids introducing artifacts, which helps accurately estimate reflection and transmission. Our experiments on real-world images with various types of reflection demonstrate the effectiveness of our model with reflection-free flash-only cues: our model outperforms state-of-the-art reflection removal approaches by more than 5.23dB in PSNR. We extend our approach to handheld photography to address the misalignment between the flash and no-flash pair. With misaligned training data and the alignment module, our aligned model outperforms our previous version by more than 3.19dB in PSNR on a misaligned dataset. We also study using linear RGB images as training data. Our source code and dataset are publicly available at https://github.com/ChenyangLEI/flash-reflection-removal.

Related papers

• Flash-Splat: 3D Reflection Removal with Flash Cues and Gaussian Splats [13.27784783829039]
  We introduce a simple yet effective approach for separating transmitted and reflected light. Our method, Flash-Splat, accurately reconstructs both transmitted and reflected scenes in 3D.
  arXiv  Detail & Related papers  (2024-10-03T17:59:59Z)
• Nighttime Smartphone Reflective Flare Removal Using Optical Center Symmetry Prior [81.64647648269889]
  Reflective flare is a phenomenon that occurs when light reflects inside lenses, causing bright spots or a "ghosting effect" in photos. We propose an optical center symmetry prior, which suggests that the reflective flare and light source are always symmetrical around the lens's optical center. We create the first reflective flare removal dataset called BracketFlare, which contains diverse and realistic reflective flare patterns.
  arXiv  Detail & Related papers  (2023-03-27T09:44:40Z)
• WildLight: In-the-wild Inverse Rendering with a Flashlight [77.31815397135381]
  We propose a practical photometric solution for in-the-wild inverse rendering under unknown ambient lighting. Our system recovers scene geometry and reflectance using only multi-view images captured by a smartphone. We demonstrate by extensive experiments that our method is easy to implement, casual to set up, and consistently outperforms existing in-the-wild inverse rendering techniques.
  arXiv  Detail & Related papers  (2023-03-24T17:59:56Z)
• Seeing Far in the Dark with Patterned Flash [5.540878289831889]
  We propose a new flash technique, named patterned flash'', for flash imaging at a long distance. Patterned flash concentrates optical power into a dot array. We develop a joint image reconstruction and depth estimation algorithm with a convolutional neural network.
  arXiv  Detail & Related papers  (2022-07-25T23:16:50Z)
• Robust Reflection Removal with Reflection-free Flash-only Cues [52.46297802064146]
  We propose a reflection-free cue for robust reflection removal from a pair of flash and ambient (no-flash) images. Our model outperforms state-of-the-art reflection removal approaches by more than 5.23dB in PSNR, 0.04 in SSIM, and 0.068 in LPIPS.
  arXiv  Detail & Related papers  (2021-03-07T05:27:43Z)
• Deep Denoising of Flash and No-Flash Pairs for Photography in Low-Light Environments [51.74566709730618]
  We introduce a neural network-based method to denoise pairs of images taken in quick succession, with and without a flash, in low-light environments. Our goal is to produce a high-quality rendering of the scene that preserves the color and mood from the ambient illumination of the noisy no-flash image.
  arXiv  Detail & Related papers  (2020-12-09T15:41:16Z)
• Polarized Reflection Removal with Perfect Alignment in the Wild [66.48211204364142]
  We present a novel formulation to removing reflection from polarized images in the wild. We first identify the misalignment issues of existing reflection removal datasets. We build a new dataset with more than 100 types of glass in which obtained transmission images are perfectly aligned with input mixed images.
  arXiv  Detail & Related papers  (2020-03-28T13:29:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.