Radiance Surfaces: Optimizing Surface Representations with a 5D Radiance Field Loss

• URL: http://arxiv.org/abs/2501.18627v2
• Date: Tue, 29 Apr 2025 14:20:57 GMT
• Title: Radiance Surfaces: Optimizing Surface Representations with a 5D Radiance Field Loss
• Authors: Ziyi Zhang, Nicolas Roussel, Thomas Müller, Tizian Zeltner, Merlin Nimier-David, Fabrice Rousselle, Wenzel Jakob,
• Abstract summary: We present a technique to convert images into a radiance surface-based scene representation.<n>We introduce a yet impactful modification of the loss function changes to only a few lines of code.<n>Our method generates explicit surfaces in place of an exponential volume.
• Score: 19.583904151537443
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We present a fast and simple technique to convert images into a radiance surface-based scene representation. Building on existing radiance volume reconstruction algorithms, we introduce a subtle yet impactful modification of the loss function requiring changes to only a few lines of code: instead of integrating the radiance field along rays and supervising the resulting images, we project the training images into the scene to directly supervise the spatio-directional radiance field. The primary outcome of this change is the complete removal of alpha blending and ray marching from the image formation model, instead moving these steps into the loss computation. In addition to promoting convergence to surfaces, this formulation assigns explicit semantic meaning to 2D subsets of the radiance field, turning them into well-defined radiance surfaces. We finally extract a level set from this representation, which results in a high-quality radiance surface model. Our method retains much of the speed and quality of the baseline algorithm. For instance, a suitably modified variant of Instant NGP maintains comparable computational efficiency, while achieving an average PSNR that is only 0.1 dB lower. Most importantly, our method generates explicit surfaces in place of an exponential volume, doing so with a level of simplicity not seen in prior work.

Related papers

• Flash Cache: Reducing Bias in Radiance Cache Based Inverse Rendering [62.92985004295714]
  We present a method that avoids approximations that introduce bias into the renderings and, more importantly, the gradients used for optimization. We show that by removing these biases our approach improves the generality of radiance cache based inverse rendering, as well as increasing quality in the presence of challenging light transport effects such as specular reflections.
  arXiv  Detail & Related papers  (2024-09-09T17:59:57Z)
• CVT-xRF: Contrastive In-Voxel Transformer for 3D Consistent Radiance Fields from Sparse Inputs [65.80187860906115]
  We propose a novel approach to improve NeRF's performance with sparse inputs. We first adopt a voxel-based ray sampling strategy to ensure that the sampled rays intersect with a certain voxel in 3D space. We then randomly sample additional points within the voxel and apply a Transformer to infer the properties of other points on each ray, which are then incorporated into the volume rendering.
  arXiv  Detail & Related papers  (2024-03-25T15:56:17Z)
• DPPE: Dense Pose Estimation in a Plenoxels Environment using Gradient Approximation [1.6512610612587693]
  We present DPPE, a dense pose estimation algorithm that functions over a Plenoxels environment. Recent advances in neural radiance field techniques have shown that it is a powerful tool for environment representation.
  arXiv  Detail & Related papers  (2024-03-16T02:22:10Z)
• Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesis [88.86777314004044]
  We propose a 3D Gaussian splatting-based framework, namely X-Gaussian, for X-ray novel view visualization. Experiments show that our X-Gaussian outperforms state-of-the-art methods by 6.5 dB while enjoying less than 15% training time and over 73x inference speed.
  arXiv  Detail & Related papers  (2024-03-07T00:12:08Z)
• BID-NeRF: RGB-D image pose estimation with inverted Neural Radiance Fields [0.0]
  We aim to improve the Inverted Neural Radiance Fields (iNeRF) algorithm which defines the image pose estimation problem as a NeRF based iterative linear optimization. NeRFs are novel neural space representation models that can synthesize photorealistic novel views of real-world scenes or objects.
  arXiv  Detail & Related papers  (2023-10-05T14:27:06Z)
• TensoIR: Tensorial Inverse Rendering [51.57268311847087]
  TensoIR is a novel inverse rendering approach based on tensor factorization and neural fields. TensoRF is a state-of-the-art approach for radiance field modeling.
  arXiv  Detail & Related papers  (2023-04-24T21:39:13Z)
• Retinexformer: One-stage Retinex-based Transformer for Low-light Image Enhancement [96.09255345336639]
  We formulate a principled One-stage Retinex-based Framework (ORF) to enhance low-light images. ORF first estimates the illumination information to light up the low-light image and then restores the corruption to produce the enhanced image. Our algorithm, Retinexformer, significantly outperforms state-of-the-art methods on thirteen benchmarks.
  arXiv  Detail & Related papers  (2023-03-12T16:54:08Z)
• Differentiable Rendering with Reparameterized Volume Sampling [2.717399369766309]
  In view synthesis, a neural radiance field approximates underlying density and radiance fields based on a sparse set of scene pictures. This rendering algorithm is fully differentiable and facilitates gradient-based optimization of the fields. We propose a simple end-to-end differentiable sampling algorithm based on inverse transform sampling.
  arXiv  Detail & Related papers  (2023-02-21T19:56:50Z)
• IBL-NeRF: Image-Based Lighting Formulation of Neural Radiance Fields [12.056350920398396]
  We propose IBL-NeRF, which decomposes the neural radiance fields (NeRF) of large-scale indoor scenes into intrinsic components. Our approach inherits superior visual quality and multi-view consistency for synthesized images as well as the intrinsic components.
  arXiv  Detail & Related papers  (2022-10-15T05:38:55Z)
• Estimating Neural Reflectance Field from Radiance Field using Tree Structures [29.431165709718794]
  We present a new method for estimating the Neural Reflectance Field (NReF) of an object from a set of posed multi-view images under unknown lighting. NReF represents 3D geometry and appearance of objects in a disentangled manner, and are hard to be estimated from images only. Our method solves this problem by exploiting the Neural Radiance Field (NeRF) as a proxy representation, from which we perform further decomposition.
  arXiv  Detail & Related papers  (2022-10-09T10:21:31Z)
• Deep Learning Adapted Acceleration for Limited-view Photoacoustic Computed Tomography [1.8830359888767887]
  Photoacoustic computed tomography (PACT) uses unfocused large-area light to illuminate the target with ultrasound transducer array for PA signal detection. Limited-view issue could cause a low-quality image in PACT due to the limitation of geometric condition. A model-based method that combines the mathematical variational model with deep learning is proposed to speed up and regularize the unrolled procedure of reconstruction.
  arXiv  Detail & Related papers  (2021-11-08T02:05:58Z)
• Generative Occupancy Fields for 3D Surface-Aware Image Synthesis [123.11969582055382]
  Generative Occupancy Fields (GOF) is a novel model based on generative radiance fields. GOF can synthesize high-quality images with 3D consistency and simultaneously learn compact and smooth object surfaces.
  arXiv  Detail & Related papers  (2021-11-01T14:20:43Z)
• Leveraging Spatial and Photometric Context for Calibrated Non-Lambertian Photometric Stereo [61.6260594326246]
  We introduce an efficient fully-convolutional architecture that can leverage both spatial and photometric context simultaneously. Using separable 4D convolutions and 2D heat-maps reduces the size and makes more efficient.
  arXiv  Detail & Related papers  (2021-03-22T18:06:58Z)
• Uncalibrated Neural Inverse Rendering for Photometric Stereo of General Surfaces [103.08512487830669]
  This paper presents an uncalibrated deep neural network framework for the photometric stereo problem. Existing neural network-based methods either require exact light directions or ground-truth surface normals of the object or both. We propose an uncalibrated neural inverse rendering approach to this problem.
  arXiv  Detail & Related papers  (2020-12-12T10:33:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.