Flash-Splat: 3D Reflection Removal with Flash Cues and Gaussian Splats

• URL: http://arxiv.org/abs/2410.02764v1
• Date: Thu, 3 Oct 2024 17:59:59 GMT
• Title: Flash-Splat: 3D Reflection Removal with Flash Cues and Gaussian Splats
• Authors: Mingyang Xie, Haoming Cai, Sachin Shah, Yiran Xu, Brandon Y. Feng, Jia-Bin Huang, Christopher A. Metzler,
• Abstract summary: We introduce a simple yet effective approach for separating transmitted and reflected light. Our method, Flash-Splat, accurately reconstructs both transmitted and reflected scenes in 3D.
• Score: 13.27784783829039
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce a simple yet effective approach for separating transmitted and reflected light. Our key insight is that the powerful novel view synthesis capabilities provided by modern inverse rendering methods (e.g.,~3D Gaussian splatting) allow one to perform flash/no-flash reflection separation using unpaired measurements -- this relaxation dramatically simplifies image acquisition over conventional paired flash/no-flash reflection separation methods. Through extensive real-world experiments, we demonstrate our method, Flash-Splat, accurately reconstructs both transmitted and reflected scenes in 3D. Our method outperforms existing 3D reflection separation methods, which do not leverage illumination control, by a large margin. Our project webpage is at https://flash-splat.github.io/.

Related papers

• IllumiNeRF: 3D Relighting Without Inverse Rendering [25.642960820693947]
  We show how to relight each input image using an image diffusion model conditioned on target environment lighting and estimated object geometry. We reconstruct a Neural Radiance Field (NeRF) with these relit images, from which we render novel views under the target lighting. We demonstrate that this strategy is surprisingly competitive and achieves state-of-the-art results on multiple relighting benchmarks.
  arXiv  Detail & Related papers  (2024-06-10T17:59:59Z)
• Towards Flexible Interactive Reflection Removal with Human Guidance [75.38207315080624]
  Single image reflection removal is inherently ambiguous, as both the reflection and transmission components requiring separation may follow natural image statistics. Existing methods attempt to address the issue by using various types of low-level and physics-based cues as sources of reflection signals. In this paper, we aim to explore a novel flexible interactive reflection removal approach that leverages various forms of sparse human guidance.
  arXiv  Detail & Related papers  (2024-06-03T17:34:37Z)
• Relightify: Relightable 3D Faces from a Single Image via Diffusion Models [86.3927548091627]
  We present the first approach to use diffusion models as a prior for highly accurate 3D facial BRDF reconstruction from a single image. In contrast to existing methods, we directly acquire the observed texture from the input image, thus, resulting in more faithful and consistent estimation.
  arXiv  Detail & Related papers  (2023-05-10T11:57:49Z)
• WildLight: In-the-wild Inverse Rendering with a Flashlight [77.31815397135381]
  We propose a practical photometric solution for in-the-wild inverse rendering under unknown ambient lighting. Our system recovers scene geometry and reflectance using only multi-view images captured by a smartphone. We demonstrate by extensive experiments that our method is easy to implement, casual to set up, and consistently outperforms existing in-the-wild inverse rendering techniques.
  arXiv  Detail & Related papers  (2023-03-24T17:59:56Z)
• Robust Reflection Removal with Flash-only Cues in the Wild [88.13531903652526]
  We propose a reflection-free cue for robust reflection removal from a pair of flash and ambient (no-flash) images. Our model outperforms state-of-the-art reflection removal approaches by more than 5.23dB in PSNR. We extend our approach to handheld photography to address the misalignment between the flash and no-flash pair.
  arXiv  Detail & Related papers  (2022-11-05T14:09:10Z)
• Seeing Far in the Dark with Patterned Flash [5.540878289831889]
  We propose a new flash technique, named patterned flash'', for flash imaging at a long distance. Patterned flash concentrates optical power into a dot array. We develop a joint image reconstruction and depth estimation algorithm with a convolutional neural network.
  arXiv  Detail & Related papers  (2022-07-25T23:16:50Z)
• Robust Reflection Removal with Reflection-free Flash-only Cues [52.46297802064146]
  We propose a reflection-free cue for robust reflection removal from a pair of flash and ambient (no-flash) images. Our model outperforms state-of-the-art reflection removal approaches by more than 5.23dB in PSNR, 0.04 in SSIM, and 0.068 in LPIPS.
  arXiv  Detail & Related papers  (2021-03-07T05:27:43Z)
• Deep Denoising of Flash and No-Flash Pairs for Photography in Low-Light Environments [51.74566709730618]
  We introduce a neural network-based method to denoise pairs of images taken in quick succession, with and without a flash, in low-light environments. Our goal is to produce a high-quality rendering of the scene that preserves the color and mood from the ambient illumination of the noisy no-flash image.
  arXiv  Detail & Related papers  (2020-12-09T15:41:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.