Ambient-robust Inverse Rendering using Active RGB-NIR Imaging

• URL: http://arxiv.org/abs/2605.30250v1
• Date: Thu, 28 May 2026 17:13:41 GMT
• Title: Ambient-robust Inverse Rendering using Active RGB-NIR Imaging
• Abstract summary: Inverse rendering aims to reconstruct geometry and reflectance of objects from images.<n>We introduce an ambient-robust inverse rendering method enabled by active RGB-NIR imaging.
• Score: 22.661549690548544
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Inverse rendering aims to reconstruct geometry and reflectance of objects from images. Despite recent progress, existing methods often produces inaccurate reconstructions that are sensitive to ambient illumination conditions. Here we introduce an ambient-robust inverse rendering method enabled by active RGB-NIR imaging. Our key insight is to leverage near-infrared (NIR) flash illumination-imperceptible to human observers-to obtain stable point-light shading that is largely invariant to ambient illumination. By using multi-view RGB images illuminated by ambient light and NIR images acquired with active NIR flash illumination, we reconstruct accurate geometry and reflectance by exploiting the complementary benefits of RGB and NIR images via a three-stage inverse rendering method. To enable dense multi-view acquisition, we develop an active imaging system equipped with a RGB-NIR camera and a NIR flash mounted on a mobile base. Using this system, we collect the first multi-view RGB-NIR inverse rendering dataset captured under multiple ambient illumination conditions. Experiments demonstrate that our method outperforms prior approaches, achieving accurate geometry and reflectance estimation across multiple ambient lighting scenarios.

Related papers

• Cyclops: LiDAR as a Camera That Dreams in Color [32.25721672776894]
  We propose a framework that translates sparse Non-Repetitive Scanning LiDAR (NRS-LiDAR) intensity into RGB video, enabling camera-free inference for all-day perception tasks.<n>Our approach first converts sparse LiDAR intensity projections into dense representations via a frozen pre-trained densification module.<n>The dense intensity latent is then transported toward the target RGB distribution through Latent Bridge Matching.
  arXiv  Detail & Related papers  (2026-08-17T08:37:09Z)
• Toward Robust and 3D-Aware RGB-NIR Imaging in the Dark [68.49637363691325]
  This paper offers a new perspective for RGB-NIR low-light imaging by incorporating 3D-aware neural modeling.<n>A powerful model can be optimized to implicitly fuse extremely noisy RGB observations with NIR cues in 3D space, effectively recovering clean RGB images.
  arXiv  Detail & Related papers  (2026-07-31T17:59:18Z)
• SEAR: Simple and Efficient Adaptation of Visual Geometric Transformers for RGB+Thermal 3D Reconstruction [28.61594700250136]
  We propose SEAR, a strategy that adapts a pretrained geometry transformer to multimodal RGB-T inputs.<n>Our approach significantly outperforms state-of-the-art methods for 3D reconstruction and camera pose estimation.<n>We validate our architecture through extensive ablation studies, demonstrating how the model aligns both modalities.
  arXiv  Detail & Related papers  (2026-03-19T11:27:21Z)
• ROGR: Relightable 3D Objects using Generative Relighting [71.35020300131261]
  We introduce ROGR, a novel approach that reconstructs a relightable 3D model of an object captured from multiple views.<n>We train a lighting-conditioned Neural Radiance Field (NeRF) that outputs the object's appearance under any input environmental lighting.<n>We evaluate our approach on the established TensoIR and Stanford-ORB datasets, and showcase our approach on real-world object captures.
  arXiv  Detail & Related papers  (2025-10-03T16:35:22Z)
• Pixel-aligned RGB-NIR Stereo Imaging and Dataset for Robot Vision [11.112091693344354]
  We introduce a robotic vision system equipped with pixel-aligned RGB-NIR stereo cameras and a LiDAR sensor mounted on a mobile robot.<n>The system simultaneously captures pixel-aligned pairs of RGB stereo images, NIR stereo images, and temporally synchronized LiDAR points.<n> Experimental results demonstrate the effectiveness of using pixel-aligned RGB-NIR images across diverse lighting conditions.
  arXiv  Detail & Related papers  (2024-11-27T03:44:21Z)
• Factored-NeuS: Reconstructing Surfaces, Illumination, and Materials of Possibly Glossy Objects [58.25772313290338]
  We develop a method that recovers the surface, materials, and illumination of a scene from its posed multi-view images.<n>It does not require any additional data and can handle glossy objects or bright lighting.
  arXiv  Detail & Related papers  (2023-05-29T07:44:19Z)
• WildLight: In-the-wild Inverse Rendering with a Flashlight [77.31815397135381]
  We propose a practical photometric solution for in-the-wild inverse rendering under unknown ambient lighting. Our system recovers scene geometry and reflectance using only multi-view images captured by a smartphone. We demonstrate by extensive experiments that our method is easy to implement, casual to set up, and consistently outperforms existing in-the-wild inverse rendering techniques.
  arXiv  Detail & Related papers  (2023-03-24T17:59:56Z)
• Visibility Constrained Wide-band Illumination Spectrum Design for Seeing-in-the-Dark [38.11468156313255]
  Seeing-in-the-dark is one of the most important and challenging computer vision tasks. In this paper, we try to robustify NIR2RGB translation by designing the optimal spectrum of auxiliary illumination in the wide-band VIS-NIR range.
  arXiv  Detail & Related papers  (2023-03-21T07:27:37Z)
• Event Fusion Photometric Stereo Network [3.0778023655689144]
  We introduce a novel method to estimate surface normal of an object in an ambient light environment using RGB and event cameras. This is the first study to use event cameras for photometric stereo in continuous light sources and ambient light environments.
  arXiv  Detail & Related papers  (2023-03-01T08:13:26Z)
• Semantic-embedded Unsupervised Spectral Reconstruction from Single RGB Images in the Wild [48.44194221801609]
  We propose a new lightweight and end-to-end learning-based framework to tackle this challenge. We progressively spread the differences between input RGB images and re-projected RGB images from recovered HS images via effective camera spectral response function estimation. Our method significantly outperforms state-of-the-art unsupervised methods and even exceeds the latest supervised method under some settings.
  arXiv  Detail & Related papers  (2021-08-15T05:19:44Z)
• PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting [60.75436852495868]
  We present PhySG, an inverse rendering pipeline that reconstructs geometry, materials, and illumination from scratch from RGB input images. We demonstrate, with both synthetic and real data, that our reconstructions not only enable rendering of novel viewpoints, but also physics-based appearance editing of materials and illumination.
  arXiv  Detail & Related papers  (2021-04-01T17:59:02Z)
• Tuning IR-cut Filter for Illumination-aware Spectral Reconstruction from RGB [84.1657998542458]
  It has been proven that the reconstruction accuracy relies heavily on the spectral response of the RGB camera in use. This paper explores the filter-array based color imaging mechanism of existing RGB cameras, and proposes to design the IR-cut filter properly for improved spectral recovery.
  arXiv  Detail & Related papers  (2021-03-26T19:42:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.