Neural Inverse Rendering from Propagating Light

• URL: http://arxiv.org/abs/2506.05347v1
• Date: Thu, 05 Jun 2025 17:59:55 GMT
• Title: Neural Inverse Rendering from Propagating Light
• Authors: Anagh Malik, Benjamin Attal, Andrew Xie, Matthew O'Toole, David B. Lindell,
• Abstract summary: We present the first system for physically based, neural inverse rendering from multi-viewpoint videos of propagating light.<n>Our approach relies on a time-resolved extension of neural radiance caching.
• Score: 17.469575212228122
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present the first system for physically based, neural inverse rendering from multi-viewpoint videos of propagating light. Our approach relies on a time-resolved extension of neural radiance caching -- a technique that accelerates inverse rendering by storing infinite-bounce radiance arriving at any point from any direction. The resulting model accurately accounts for direct and indirect light transport effects and, when applied to captured measurements from a flash lidar system, enables state-of-the-art 3D reconstruction in the presence of strong indirect light. Further, we demonstrate view synthesis of propagating light, automatic decomposition of captured measurements into direct and indirect components, as well as novel capabilities such as multi-view time-resolved relighting of captured scenes.

Related papers

• Transientangelo: Few-Viewpoint Surface Reconstruction Using Single-Photon Lidar [8.464054039931245]
  Lidar captures 3D scene geometry by emitting pulses of light to a target and recording the speed-of-light time delay of the reflected light.<n> conventional lidar systems do not output the raw, captured waveforms of backscattered light.<n>We develop new regularization strategies that improve robustness to photon noise, enabling accurate surface reconstruction with as few as 10 photons per pixel.
  arXiv  Detail & Related papers  (2024-08-22T08:12:09Z)
• Flying with Photons: Rendering Novel Views of Propagating Light [37.06220870989172]
  We present an imaging and neural rendering technique that seeks to synthesize videos of light propagating through a scene from novel, moving camera viewpoints. Our approach relies on a new ultrafast imaging setup to capture a first-of-its kind, multi-viewpoint video dataset with pico-second-level temporal resolution.
  arXiv  Detail & Related papers  (2024-04-09T17:48:52Z)
• GIR: 3D Gaussian Inverse Rendering for Relightable Scene Factorization [62.13932669494098]
  This paper presents a 3D Gaussian Inverse Rendering (GIR) method, employing 3D Gaussian representations to factorize the scene into material properties, light, and geometry. We compute the normal of each 3D Gaussian using the shortest eigenvector, with a directional masking scheme forcing accurate normal estimation without external supervision. We adopt an efficient voxel-based indirect illumination tracing scheme that stores direction-aware outgoing radiance in each 3D Gaussian to disentangle secondary illumination for approximating multi-bounce light transport.
  arXiv  Detail & Related papers  (2023-12-08T16:05:15Z)
• Self-Calibrating, Fully Differentiable NLOS Inverse Rendering [15.624750787186803]
  Time-resolved non-line-of-sight (NLOS) imaging methods reconstruct hidden scenes by inverting the optical paths of indirect illumination measured at visible relay surfaces. We introduce a fully-differentiable end-to-end NLOS inverse rendering pipeline that self-calibrates the imaging parameters during the reconstruction of hidden scenes. We demonstrate the robustness of our method to consistently reconstruct geometry and albedo, even under significant noise levels.
  arXiv  Detail & Related papers  (2023-09-21T13:15:54Z)
• 3D Motion Magnification: Visualizing Subtle Motions with Time Varying Radiance Fields [58.6780687018956]
  We present a 3D motion magnification method that can magnify subtle motions from scenes captured by a moving camera. We represent the scene with time-varying radiance fields and leverage the Eulerian principle for motion magnification. We evaluate the effectiveness of our method on both synthetic and real-world scenes captured under various camera setups.
  arXiv  Detail & Related papers  (2023-08-07T17:59:59Z)
• TensoIR: Tensorial Inverse Rendering [51.57268311847087]
  TensoIR is a novel inverse rendering approach based on tensor factorization and neural fields. TensoRF is a state-of-the-art approach for radiance field modeling.
  arXiv  Detail & Related papers  (2023-04-24T21:39:13Z)
• Neural 3D Reconstruction in the Wild [86.6264706256377]
  We introduce a new method that enables efficient and accurate surface reconstruction from Internet photo collections. We present a new benchmark and protocol for evaluating reconstruction performance on such in-the-wild scenes.
  arXiv  Detail & Related papers  (2022-05-25T17:59:53Z)
• Modeling Indirect Illumination for Inverse Rendering [31.734819333921642]
  In this paper, we propose a novel approach to efficiently recovering spatially-varying indirect illumination. The key insight is that indirect illumination can be conveniently derived from the neural radiance field learned from input images. Experiments on both synthetic and real data demonstrate the superior performance of our approach compared to previous work.
  arXiv  Detail & Related papers  (2022-04-14T09:10:55Z)
• Learning Neural Transmittance for Efficient Rendering of Reflectance Fields [43.24427791156121]
  We propose a novel method based on precomputed Neural Transmittance Functions to accelerate rendering of neural reflectance fields. Results on real and synthetic scenes demonstrate almost two order of magnitude speedup for renderings under environment maps with minimal accuracy loss.
  arXiv  Detail & Related papers  (2021-10-25T21:12:25Z)
• Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering [60.02806355570514]
  Inferring representations of 3D scenes from 2D observations is a fundamental problem of computer graphics, computer vision, and artificial intelligence. We propose a novel neural scene representation, Light Field Networks or LFNs, which represent both geometry and appearance of the underlying 3D scene in a 360-degree, four-dimensional light field. Rendering a ray from an LFN requires only a *single* network evaluation, as opposed to hundreds of evaluations per ray for ray-marching or based on volumetrics.
  arXiv  Detail & Related papers  (2021-06-04T17:54:49Z)
• MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo [52.329580781898116]
  We present MVSNeRF, a novel neural rendering approach that can efficiently reconstruct neural radiance fields for view synthesis. Unlike prior works on neural radiance fields that consider per-scene optimization on densely captured images, we propose a generic deep neural network that can reconstruct radiance fields from only three nearby input views via fast network inference.
  arXiv  Detail & Related papers  (2021-03-29T13:15:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.