Cross-Spectral Neural Radiance Fields

• URL: http://arxiv.org/abs/2209.00648v1
• Date: Thu, 1 Sep 2022 17:59:58 GMT
• Title: Cross-Spectral Neural Radiance Fields
• Authors: Matteo Poggi, Pierluigi Zama Ramirez, Fabio Tosi, Samuele Salti, Stefano Mattoccia, Luigi Di Stefano
• Abstract summary: We propose X-NeRF, a novel method to learn a Cross-Spectral scene representation given images captured from cameras with different light spectrum sensitivity. X-NeRF exploits Normalized Cross-Device Coordinates (NXDC) to render images of different modalities from arbitrary viewpoints.
• Score: 49.28588927121722
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose X-NeRF, a novel method to learn a Cross-Spectral scene representation given images captured from cameras with different light spectrum sensitivity, based on the Neural Radiance Fields formulation. X-NeRF optimizes camera poses across spectra during training and exploits Normalized Cross-Device Coordinates (NXDC) to render images of different modalities from arbitrary viewpoints, which are aligned and at the same resolution. Experiments on 16 forward-facing scenes, featuring color, multi-spectral and infrared images, confirm the effectiveness of X-NeRF at modeling Cross-Spectral scene representations.

Related papers

• Towards virtual painting recolouring using Vision Transformer on X-Ray Fluorescence datacubes [80.32085982862151]
  We define a pipeline to perform virtual painting recolouring using raw data of X-Ray Fluorescence (XRF) analysis on pictorial artworks. To circumvent the small dataset size, we generate a synthetic dataset, starting from a database of XRF spectra. We define a Deep Variational Embedding network to embed the XRF spectra into a lower dimensional, K-Means friendly, metric space.
  arXiv  Detail & Related papers  (2024-10-11T14:05:28Z)
• TeX-NeRF: Neural Radiance Fields from Pseudo-TeX Vision [5.77388464529179]
  We propose Ne-RF, a 3D reconstruction method using only infrared images. We map the temperatures, emissivities (e), and textures (X) of the scene into the saturation (S), hue (H), and value (V) channels of the color space. Novel view synthesis using the processed images has yielded excellent results.
  arXiv  Detail & Related papers  (2024-10-07T09:43:28Z)
• NeRF-Casting: Improved View-Dependent Appearance with Consistent Reflections [57.63028964831785]
  Recent works have improved NeRF's ability to render detailed specular appearance of distant environment illumination, but are unable to synthesize consistent reflections of closer content. We address these issues with an approach based on ray tracing. Instead of querying an expensive neural network for the outgoing view-dependent radiance at points along each camera ray, our model casts rays from these points and traces them through the NeRF representation to render feature vectors.
  arXiv  Detail & Related papers  (2024-05-23T17:59:57Z)
• SpectralNeRF: Physically Based Spectral Rendering with Neural Radiance Field [70.15900280156262]
  We propose an end-to-end Neural Radiance Field (NeRF)-based architecture for high-quality physically based rendering from a novel spectral perspective. SpectralNeRF is superior to recent NeRF-based methods when synthesizing new views on synthetic and real datasets.
  arXiv  Detail & Related papers  (2023-12-14T07:19:31Z)
• Spec-NeRF: Multi-spectral Neural Radiance Fields [9.242830798112855]
  We propose Multi-spectral Neural Radiance Fields(Spec-NeRF) for jointly reconstructing a multispectral radiance field and spectral sensitivity functions(SSFs) of the camera from a set of color images filtered by different filters. Our experiments on both synthetic and real scenario datasets demonstrate that utilizing filtered RGB images with learnable NeRF and SSFs can achieve high fidelity and promising spectral reconstruction.
  arXiv  Detail & Related papers  (2023-09-14T16:17:55Z)
• Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields [40.72851892972173]
  We introduce Ref-NeRF, which replaces NeRF's parameterization of view-dependent outgoing radiance with a representation of reflected radiance and structures. We show that our model's internal representation of outgoing radiance is interpretable and useful for scene editing.
  arXiv  Detail & Related papers  (2021-12-07T18:58:37Z)
• HDR-NeRF: High Dynamic Range Neural Radiance Fields [70.80920996881113]
  We present High Dynamic Range Neural Radiance Fields (-NeRF) to recover an HDR radiance field from a set of low dynamic range (LDR) views with different exposures. We are able to generate both novel HDR views and novel LDR views under different exposures.
  arXiv  Detail & Related papers  (2021-11-29T11:06:39Z)
• NeRF++: Analyzing and Improving Neural Radiance Fields [117.73411181186088]
  Neural Radiance Fields (NeRF) achieve impressive view synthesis results for a variety of capture settings. NeRF fits multi-layer perceptrons representing view-invariant opacity and view-dependent color volumes to a set of training images. We address a parametrization issue involved in applying NeRF to 360 captures of objects within large-scale, 3D scenes.
  arXiv  Detail & Related papers  (2020-10-15T03:24:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.