Spec-NeRF: Multi-spectral Neural Radiance Fields

• URL: http://arxiv.org/abs/2310.12987v1
• Date: Thu, 14 Sep 2023 16:17:55 GMT
• Title: Spec-NeRF: Multi-spectral Neural Radiance Fields
• Authors: Jiabao Li, Yuqi Li, Ciliang Sun, Chong Wang, Jinhui Xiang
• Abstract summary: We propose Multi-spectral Neural Radiance Fields(Spec-NeRF) for jointly reconstructing a multispectral radiance field and spectral sensitivity functions(SSFs) of the camera from a set of color images filtered by different filters. Our experiments on both synthetic and real scenario datasets demonstrate that utilizing filtered RGB images with learnable NeRF and SSFs can achieve high fidelity and promising spectral reconstruction.
• Score: 9.242830798112855
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose Multi-spectral Neural Radiance Fields(Spec-NeRF) for jointly reconstructing a multispectral radiance field and spectral sensitivity functions(SSFs) of the camera from a set of color images filtered by different filters. The proposed method focuses on modeling the physical imaging process, and applies the estimated SSFs and radiance field to synthesize novel views of multispectral scenes. In this method, the data acquisition requires only a low-cost trichromatic camera and several off-the-shelf color filters, making it more practical than using specialized 3D scanning and spectral imaging equipment. Our experiments on both synthetic and real scenario datasets demonstrate that utilizing filtered RGB images with learnable NeRF and SSFs can achieve high fidelity and promising spectral reconstruction while retaining the inherent capability of NeRF to comprehend geometric structures. Code is available at https://github.com/CPREgroup/SpecNeRF-v2.

Related papers

• Hyperspectral Neural Radiance Fields [11.485829401765521]
  We propose a hyperspectral 3D reconstruction using Neural Radiance Fields (NeRFs) NeRFs have seen widespread success in creating high quality volumetric 3D representations of scenes captured by a variety of camera models. We show that our hyperspectral NeRF approach enables creating fast, accurate volumetric 3D hyperspectral scenes.
  arXiv  Detail & Related papers  (2024-03-21T21:18:08Z)
• SpectralNeRF: Physically Based Spectral Rendering with Neural Radiance Field [70.15900280156262]
  We propose an end-to-end Neural Radiance Field (NeRF)-based architecture for high-quality physically based rendering from a novel spectral perspective. SpectralNeRF is superior to recent NeRF-based methods when synthesizing new views on synthetic and real datasets.
  arXiv  Detail & Related papers  (2023-12-14T07:19:31Z)
• Multi-Space Neural Radiance Fields [74.46513422075438]
  Existing Neural Radiance Fields (NeRF) methods suffer from the existence of reflective objects. We propose a multi-space neural radiance field (MS-NeRF) that represents the scene using a group of feature fields in parallel sub-spaces. Our approach significantly outperforms the existing single-space NeRF methods for rendering high-quality scenes.
  arXiv  Detail & Related papers  (2023-05-07T13:11:07Z)
• Longwave infrared multispectral image sensor system using aluminum-germanium plasmonic filter arrays [0.8081564951955755]
  A multispectral camera records image data in various wavelengths across the electromagnetic spectrum to acquire additional information that a conventional camera fails to capture. We experimentally demonstrate an LWIR multispectral image sensor with three wavelength bands using optical elements made of an aluminum-based plasmonic filter array sandwiched in germanium. Our work demonstrates a versatile spectral thermography technique for detecting target signatures in the LWIR range and other advanced spectral analyses.
  arXiv  Detail & Related papers  (2023-03-03T01:47:32Z)
• Cross-Spectral Neural Radiance Fields [49.28588927121722]
  We propose X-NeRF, a novel method to learn a Cross-Spectral scene representation given images captured from cameras with different light spectrum sensitivity. X-NeRF exploits Normalized Cross-Device Coordinates (NXDC) to render images of different modalities from arbitrary viewpoints.
  arXiv  Detail & Related papers  (2022-09-01T17:59:58Z)
• NeRF-SR: High-Quality Neural Radiance Fields using Super-Sampling [82.99453001445478]
  We present NeRF-SR, a solution for high-resolution (HR) novel view synthesis with mostly low-resolution (LR) inputs. Our method is built upon Neural Radiance Fields (NeRF) that predicts per-point density and color with a multi-layer perceptron.
  arXiv  Detail & Related papers  (2021-12-03T07:33:47Z)
• Tuning IR-cut Filter for Illumination-aware Spectral Reconstruction from RGB [84.1657998542458]
  It has been proven that the reconstruction accuracy relies heavily on the spectral response of the RGB camera in use. This paper explores the filter-array based color imaging mechanism of existing RGB cameras, and proposes to design the IR-cut filter properly for improved spectral recovery.
  arXiv  Detail & Related papers  (2021-03-26T19:42:21Z)
• NeRF++: Analyzing and Improving Neural Radiance Fields [117.73411181186088]
  Neural Radiance Fields (NeRF) achieve impressive view synthesis results for a variety of capture settings. NeRF fits multi-layer perceptrons representing view-invariant opacity and view-dependent color volumes to a set of training images. We address a parametrization issue involved in applying NeRF to 360 captures of objects within large-scale, 3D scenes.
  arXiv  Detail & Related papers  (2020-10-15T03:24:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.