Flash-Split: 2D Reflection Removal with Flash Cues and Latent Diffusion Separation

• URL: http://arxiv.org/abs/2501.00637v1
• Date: Tue, 31 Dec 2024 20:27:23 GMT
• Title: Flash-Split: 2D Reflection Removal with Flash Cues and Latent Diffusion Separation
• Authors: Tianfu Wang, Mingyang Xie, Haoming Cai, Sachin Shah, Christopher A. Metzler,
• Abstract summary: Flash-Split is a framework for separating transmitted and reflected light using a single (potentially misaligned) pair of flash/no-flash images. By validating Flash-Split on challenging real-world scenes, we demonstrate state-of-the-art reflection separation performance.
• Score: 15.530943752811929
• License:
• Abstract: Transparent surfaces, such as glass, create complex reflections that obscure images and challenge downstream computer vision applications. We introduce Flash-Split, a robust framework for separating transmitted and reflected light using a single (potentially misaligned) pair of flash/no-flash images. Our core idea is to perform latent-space reflection separation while leveraging the flash cues. Specifically, Flash-Split consists of two stages. Stage 1 separates apart the reflection latent and transmission latent via a dual-branch diffusion model conditioned on an encoded flash/no-flash latent pair, effectively mitigating the flash/no-flash misalignment issue. Stage 2 restores high-resolution, faithful details to the separated latents, via a cross-latent decoding process conditioned on the original images before separation. By validating Flash-Split on challenging real-world scenes, we demonstrate state-of-the-art reflection separation performance and significantly outperform the baseline methods.

Related papers

• Flash-Splat: 3D Reflection Removal with Flash Cues and Gaussian Splats [13.27784783829039]
  We introduce a simple yet effective approach for separating transmitted and reflected light. Our method, Flash-Splat, accurately reconstructs both transmitted and reflected scenes in 3D.
  arXiv  Detail & Related papers  (2024-10-03T17:59:59Z)
• DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior [70.46245698746874]
  We present DiffBIR, a general restoration pipeline that could handle different blind image restoration tasks. DiffBIR decouples blind image restoration problem into two stages: 1) degradation removal: removing image-independent content; 2) information regeneration: generating the lost image content. In the first stage, we use restoration modules to remove degradations and obtain high-fidelity restored results. For the second stage, we propose IRControlNet that leverages the generative ability of latent diffusion models to generate realistic details.
  arXiv  Detail & Related papers  (2023-08-29T07:11:52Z)
• Dual Degradation Representation for Joint Deraining and Low-Light Enhancement in the Dark [57.85378202032541]
  Rain in the dark poses a significant challenge to deploying real-world applications such as autonomous driving, surveillance systems, and night photography. Existing low-light enhancement or deraining methods struggle to brighten low-light conditions and remove rain simultaneously. We introduce an end-to-end model called L$2$RIRNet, designed to manage both low-light enhancement and deraining in real-world settings.
  arXiv  Detail & Related papers  (2023-05-06T10:17:42Z)
• WildLight: In-the-wild Inverse Rendering with a Flashlight [77.31815397135381]
  We propose a practical photometric solution for in-the-wild inverse rendering under unknown ambient lighting. Our system recovers scene geometry and reflectance using only multi-view images captured by a smartphone. We demonstrate by extensive experiments that our method is easy to implement, casual to set up, and consistently outperforms existing in-the-wild inverse rendering techniques.
  arXiv  Detail & Related papers  (2023-03-24T17:59:56Z)
• Robust Reflection Removal with Flash-only Cues in the Wild [88.13531903652526]
  We propose a reflection-free cue for robust reflection removal from a pair of flash and ambient (no-flash) images. Our model outperforms state-of-the-art reflection removal approaches by more than 5.23dB in PSNR. We extend our approach to handheld photography to address the misalignment between the flash and no-flash pair.
  arXiv  Detail & Related papers  (2022-11-05T14:09:10Z)
• Seeing Far in the Dark with Patterned Flash [5.540878289831889]
  We propose a new flash technique, named patterned flash'', for flash imaging at a long distance. Patterned flash concentrates optical power into a dot array. We develop a joint image reconstruction and depth estimation algorithm with a convolutional neural network.
  arXiv  Detail & Related papers  (2022-07-25T23:16:50Z)
• Robust Reflection Removal with Reflection-free Flash-only Cues [52.46297802064146]
  We propose a reflection-free cue for robust reflection removal from a pair of flash and ambient (no-flash) images. Our model outperforms state-of-the-art reflection removal approaches by more than 5.23dB in PSNR, 0.04 in SSIM, and 0.068 in LPIPS.
  arXiv  Detail & Related papers  (2021-03-07T05:27:43Z)
• Deep Denoising of Flash and No-Flash Pairs for Photography in Low-Light Environments [51.74566709730618]
  We introduce a neural network-based method to denoise pairs of images taken in quick succession, with and without a flash, in low-light environments. Our goal is to produce a high-quality rendering of the scene that preserves the color and mood from the ambient illumination of the noisy no-flash image.
  arXiv  Detail & Related papers  (2020-12-09T15:41:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.