MS-NeRF: Multi-Space Neural Radiance Fields

• URL: http://arxiv.org/abs/2305.04268v2
• Date: Tue, 25 Mar 2025 06:26:30 GMT
• Title: MS-NeRF: Multi-Space Neural Radiance Fields
• Authors: Ze-Xin Yin, Peng-Yi Jiao, Jiaxiong Qiu, Ming-Ming Cheng, Bo Ren,
• Abstract summary: Existing Neural Radiance Fields (NeRF) methods suffer from the existence of reflective objects, often resulting in blurry rendering.<n>We propose a multi-space neural radiance field (MS-NeRF) that represents the scene using a group of feature fields in parallel sub-spaces.<n>Our approach significantly outperforms the existing single-space NeRF methods for rendering high-quality scenes.
• Score: 48.0367339199913
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Existing Neural Radiance Fields (NeRF) methods suffer from the existence of reflective objects, often resulting in blurry or distorted rendering. Instead of calculating a single radiance field, we propose a multi-space neural radiance field (MS-NeRF) that represents the scene using a group of feature fields in parallel sub-spaces, which leads to a better understanding of the neural network toward the existence of reflective and refractive objects. Our multi-space scheme works as an enhancement to existing NeRF methods, with only small computational overheads needed for training and inferring the extra-space outputs. We design different multi-space modules for representative MLP-based and grid-based NeRF methods, which improve Mip-NeRF 360 by 4.15 dB in PSNR with 0.5% extra parameters and further improve TensoRF by 2.71 dB with 0.046% extra parameters on reflective regions without degrading the rendering quality on other regions. We further construct a novel dataset consisting of 33 synthetic scenes and 7 real captured scenes with complex reflection and refraction, where we design complex camera paths to fully benchmark the robustness of NeRF-based methods. Extensive experiments show that our approach significantly outperforms the existing single-space NeRF methods for rendering high-quality scenes concerned with complex light paths through mirror-like objects. The source code, dataset, and results are available via our project page: https://zx-yin.github.io/msnerf/.

Related papers

• Neural Reflectance Fields for Radio-Frequency Ray Tracing [12.517163884907433]
  Ray tracing is widely employed to model the propagation of radio-frequency (RF) signal in complex environment. We tackle this problem by learning the material reflectivity efficiently from the path loss of the RF signal from transmitters to receivers. We achieve this by translating the neural reflectance field from optics to RF domain by modelling both the amplitude and phase of RF signals.
  arXiv  Detail & Related papers  (2025-01-05T06:52:35Z)
• NeRF-Casting: Improved View-Dependent Appearance with Consistent Reflections [57.63028964831785]
  Recent works have improved NeRF's ability to render detailed specular appearance of distant environment illumination, but are unable to synthesize consistent reflections of closer content. We address these issues with an approach based on ray tracing. Instead of querying an expensive neural network for the outgoing view-dependent radiance at points along each camera ray, our model casts rays from these points and traces them through the NeRF representation to render feature vectors.
  arXiv  Detail & Related papers  (2024-05-23T17:59:57Z)
• Mesh2NeRF: Direct Mesh Supervision for Neural Radiance Field Representation and Generation [51.346733271166926]
  Mesh2NeRF is an approach to derive ground-truth radiance fields from textured meshes for 3D generation tasks. We validate the effectiveness of Mesh2NeRF across various tasks.
  arXiv  Detail & Related papers  (2024-03-28T11:22:53Z)
• Enhance-NeRF: Multiple Performance Evaluation for Neural Radiance Fields [2.5432277893532116]
  Neural Radiance Fields (NeRF) can generate realistic images from any viewpoint. NeRF-based models are susceptible to interference issues caused by colored "fog" noise. Our approach, coined Enhance-NeRF, adopts joint color to balance low and high reflectivity objects display.
  arXiv  Detail & Related papers  (2023-06-08T15:49:30Z)
• Learning Neural Duplex Radiance Fields for Real-Time View Synthesis [33.54507228895688]
  We propose a novel approach to distill and bake NeRFs into highly efficient mesh-based neural representations. We demonstrate the effectiveness and superiority of our approach via extensive experiments on a range of standard datasets.
  arXiv  Detail & Related papers  (2023-04-20T17:59:52Z)
• AligNeRF: High-Fidelity Neural Radiance Fields via Alignment-Aware Training [100.33713282611448]
  We conduct the first pilot study on training NeRF with high-resolution data. We propose the corresponding solutions, including marrying the multilayer perceptron with convolutional layers. Our approach is nearly free without introducing obvious training/testing costs.
  arXiv  Detail & Related papers  (2022-11-17T17:22:28Z)
• R2L: Distilling Neural Radiance Field to Neural Light Field for Efficient Novel View Synthesis [76.07010495581535]
  Rendering a single pixel requires querying the Neural Radiance Field network hundreds of times. NeLF presents a more straightforward representation over NeRF in novel view. We show the key to successfully learning a deep NeLF network is to have sufficient data.
  arXiv  Detail & Related papers  (2022-03-31T17:57:05Z)
• NeRF-SR: High-Quality Neural Radiance Fields using Super-Sampling [82.99453001445478]
  We present NeRF-SR, a solution for high-resolution (HR) novel view synthesis with mostly low-resolution (LR) inputs. Our method is built upon Neural Radiance Fields (NeRF) that predicts per-point density and color with a multi-layer perceptron.
  arXiv  Detail & Related papers  (2021-12-03T07:33:47Z)
• Learning Neural Light Fields with Ray-Space Embedding Networks [51.88457861982689]
  We propose a novel neural light field representation that is compact and directly predicts integrated radiance along rays. Our method achieves state-of-the-art quality on dense forward-facing datasets such as the Stanford Light Field dataset.
  arXiv  Detail & Related papers  (2021-12-02T18:59:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.