TraM-NeRF: Tracing Mirror and Near-Perfect Specular Reflections through
Neural Radiance Fields
- URL: http://arxiv.org/abs/2310.10650v1
- Date: Mon, 16 Oct 2023 17:59:56 GMT
- Title: TraM-NeRF: Tracing Mirror and Near-Perfect Specular Reflections through
Neural Radiance Fields
- Authors: Leif Van Holland, Ruben Bliersbach, Jan U. M\"uller, Patrick Stotko,
Reinhard Klein
- Abstract summary: Implicit representations like Neural Radiance Fields (NeRF) showed impressive results for rendering of complex scenes with fine details.
We present a novel reflection tracing method tailored for the involved volume rendering within NeRF.
We derive efficient strategies for importance sampling and the transmittance computation along rays from only few samples.
- Score: 3.061835990893184
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Implicit representations like Neural Radiance Fields (NeRF) showed impressive
results for photorealistic rendering of complex scenes with fine details.
However, ideal or near-perfectly specular reflecting objects such as mirrors,
which are often encountered in various indoor scenes, impose ambiguities and
inconsistencies in the representation of the reconstructed scene leading to
severe artifacts in the synthesized renderings. In this paper, we present a
novel reflection tracing method tailored for the involved volume rendering
within NeRF that takes these mirror-like objects into account while avoiding
the cost of straightforward but expensive extensions through standard path
tracing. By explicitly modeling the reflection behavior using physically
plausible materials and estimating the reflected radiance with Monte-Carlo
methods within the volume rendering formulation, we derive efficient strategies
for importance sampling and the transmittance computation along rays from only
few samples. We show that our novel method enables the training of consistent
representations of such challenging scenes and achieves superior results in
comparison to previous state-of-the-art approaches.
Related papers
- NeRFs are Mirror Detectors: Using Structural Similarity for Multi-View Mirror Scene Reconstruction with 3D Surface Primitives [7.116175288307167]
We present NeRF-MD, a method which shows that NeRFs can be considered as mirror detectors.
We first compute an initial estimate of the scene geometry by training a standard NeRF.
We then jointly optimize the radiance field and mirror geometry in a second training stage to refine their quality.
arXiv Detail & Related papers (2025-01-07T18:59:53Z) - Neural Reflectance Fields for Radio-Frequency Ray Tracing [12.517163884907433]
Ray tracing is widely employed to model the propagation of radio-frequency (RF) signal in complex environment.
We tackle this problem by learning the material reflectivity efficiently from the path loss of the RF signal from transmitters to receivers.
We achieve this by translating the neural reflectance field from optics to RF domain by modelling both the amplitude and phase of RF signals.
arXiv Detail & Related papers (2025-01-05T06:52:35Z) - EnvGS: Modeling View-Dependent Appearance with Environment Gaussian [78.74634059559891]
EnvGS is a novel approach that employs a set of Gaussian primitives as an explicit 3D representation for capturing reflections of environments.
To efficiently render these environment Gaussian primitives, we developed a ray-tracing-based reflection that leverages the GPU's RT core for fast rendering.
Results from multiple real-world and synthetic datasets demonstrate that our method produces significantly more detailed reflections.
arXiv Detail & Related papers (2024-12-19T18:59:57Z) - Planar Reflection-Aware Neural Radiance Fields [32.709468082010126]
We introduce a reflection-aware NeRF that jointly models planar reflectors, such as windows, and explicitly casts reflected rays to capture the source of the high-frequency reflections.
Rendering along the primary ray results in a clean, reflection-free view, while explicitly rendering along the reflected ray allows us to reconstruct highly detailed reflections.
arXiv Detail & Related papers (2024-11-07T18:55:08Z) - NeRF-Casting: Improved View-Dependent Appearance with Consistent Reflections [57.63028964831785]
Recent works have improved NeRF's ability to render detailed specular appearance of distant environment illumination, but are unable to synthesize consistent reflections of closer content.
We address these issues with an approach based on ray tracing.
Instead of querying an expensive neural network for the outgoing view-dependent radiance at points along each camera ray, our model casts rays from these points and traces them through the NeRF representation to render feature vectors.
arXiv Detail & Related papers (2024-05-23T17:59:57Z) - REFRAME: Reflective Surface Real-Time Rendering for Mobile Devices [51.983541908241726]
This work tackles the challenging task of achieving real-time novel view synthesis for reflective surfaces across various scenes.
Existing real-time rendering methods, especially those based on meshes, often have subpar performance in modeling surfaces with rich view-dependent appearances.
We decompose the color into diffuse and specular, and model the specular color in the reflected direction based on a neural environment map.
arXiv Detail & Related papers (2024-03-25T07:07:50Z) - TensoIR: Tensorial Inverse Rendering [51.57268311847087]
TensoIR is a novel inverse rendering approach based on tensor factorization and neural fields.
TensoRF is a state-of-the-art approach for radiance field modeling.
arXiv Detail & Related papers (2023-04-24T21:39:13Z) - Neural Microfacet Fields for Inverse Rendering [54.15870869037466]
We present a method for recovering materials, geometry, and environment illumination from images of a scene.
Our method uses a microfacet reflectance model within a volumetric setting by treating each sample along the ray as a (potentially non-opaque) surface.
arXiv Detail & Related papers (2023-03-31T05:38:13Z) - NeRFReN: Neural Radiance Fields with Reflections [16.28256369376256]
We introduce NeRFReN, which is built upon NeRF to model scenes with reflections.
We propose to split a scene into transmitted and reflected components, and model the two components with separate neural radiance fields.
Experiments on various self-captured scenes show that our method achieves high-quality novel view synthesis and physically sound depth estimation results.
arXiv Detail & Related papers (2021-11-30T09:36:00Z) - Neural Reflectance Fields for Appearance Acquisition [61.542001266380375]
We present Neural Reflectance Fields, a novel deep scene representation that encodes volume density, normal and reflectance properties at any 3D point in a scene.
We combine this representation with a physically-based differentiable ray marching framework that can render images from a neural reflectance field under any viewpoint and light.
arXiv Detail & Related papers (2020-08-09T22:04:36Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.