Estimating 3D Uncertainty Field: Quantifying Uncertainty for Neural Radiance Fields

• URL: http://arxiv.org/abs/2311.01815v2
• Date: Sun, 26 Nov 2023 03:44:36 GMT
• Title: Estimating 3D Uncertainty Field: Quantifying Uncertainty for Neural Radiance Fields
• Authors: Jianxiong Shen and Ruijie Ren and Adria Ruiz and Francesc Moreno-Noguer
• Abstract summary: We propose a novel approach to estimate a 3D Uncertainty Field based on the learned incomplete scene geometry. By considering the accumulated transmittance along each camera ray, our Uncertainty Field infers 2D pixel-wise uncertainty. Our experiments demonstrate that our approach is the only one that can explicitly reason about high uncertainty both on 3D unseen regions and its involved 2D rendered pixels.
• Score: 25.300284510832974
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Current methods based on Neural Radiance Fields (NeRF) significantly lack the capacity to quantify uncertainty in their predictions, particularly on the unseen space including the occluded and outside scene content. This limitation hinders their extensive applications in robotics, where the reliability of model predictions has to be considered for tasks such as robotic exploration and planning in unknown environments. To address this, we propose a novel approach to estimate a 3D Uncertainty Field based on the learned incomplete scene geometry, which explicitly identifies these unseen regions. By considering the accumulated transmittance along each camera ray, our Uncertainty Field infers 2D pixel-wise uncertainty, exhibiting high values for rays directly casting towards occluded or outside the scene content. To quantify the uncertainty on the learned surface, we model a stochastic radiance field. Our experiments demonstrate that our approach is the only one that can explicitly reason about high uncertainty both on 3D unseen regions and its involved 2D rendered pixels, compared with recent methods. Furthermore, we illustrate that our designed uncertainty field is ideally suited for real-world robotics tasks, such as next-best-view selection.

Related papers

• Manifold Sampling for Differentiable Uncertainty in Radiance Fields [82.17927517146929]
  We propose a versatile approach for learning Gaussian radiance fields with explicit and fine-grained uncertainty estimates. We demonstrate state-of-the-art performance on next-best-view planning tasks.
  arXiv  Detail & Related papers  (2024-09-19T11:22:20Z)
• Sources of Uncertainty in 3D Scene Reconstruction [19.807599821939633]
  We introduce a taxonomy that categorizes different sources of uncertainty inherent in 3D scene reconstruction methods. We extend NeRF- and GS-based methods with uncertainty estimation techniques, including learning uncertainty outputs and ensembles. Our study highlights the need for addressing various uncertainty aspects when designing NeRF/GS-based methods for uncertainty-aware 3D reconstruction.
  arXiv  Detail & Related papers  (2024-09-10T10:43:53Z)
• Calib3D: Calibrating Model Preferences for Reliable 3D Scene Understanding [55.32861154245772]
  Calib3D is a pioneering effort to benchmark and scrutinize the reliability of 3D scene understanding models. We evaluate 28 state-of-the-art models across 10 diverse 3D datasets. We introduce DeptS, a novel depth-aware scaling approach aimed at enhancing 3D model calibration.
  arXiv  Detail & Related papers  (2024-03-25T17:59:59Z)
• FisherRF: Active View Selection and Uncertainty Quantification for Radiance Fields using Fisher Information [32.66184501415286]
  This study addresses the problem of active view selection and uncertainty quantification within the domain of Radiance Fields. NeRF have greatly advanced image rendering and reconstruction, but the limited availability of 2D images poses uncertainties. By leveraging Fisher Information, we efficiently quantify observed information within Radiance Fields without ground truth data.
  arXiv  Detail & Related papers  (2023-11-29T18:20:16Z)
• GUPNet++: Geometry Uncertainty Propagation Network for Monocular 3D Object Detection [95.8940731298518]
  We propose a novel Geometry Uncertainty Propagation Network (GUPNet++) It models the uncertainty propagation relationship of the geometry projection during training, improving the stability and efficiency of the end-to-end model learning. Experiments show that the proposed approach not only obtains (state-of-the-art) SOTA performance in image-based monocular 3D detection but also demonstrates superiority in efficacy with a simplified framework.
  arXiv  Detail & Related papers  (2023-10-24T08:45:15Z)
• Safe motion planning with environment uncertainty [1.4824891788575418]
  We present an approach for safe motion planning under robot state and environment uncertainties. We first develop an approach that accounts for the landmark uncertainties during robot localization. We then extend the state-of-the-art by computing an exact expression for the collision probability.
  arXiv  Detail & Related papers  (2023-05-10T09:29:41Z)
• Uncertainty-Aware AB3DMOT by Variational 3D Object Detection [74.8441634948334]
  Uncertainty estimation is an effective tool to provide statistically accurate predictions. In this paper, we propose a Variational Neural Network-based TANet 3D object detector to generate 3D object detections with uncertainty.
  arXiv  Detail & Related papers  (2023-02-12T14:30:03Z)
• Autoregressive Uncertainty Modeling for 3D Bounding Box Prediction [63.3021778885906]
  3D bounding boxes are a widespread intermediate representation in many computer vision applications. We propose methods for leveraging our autoregressive model to make high confidence predictions and meaningful uncertainty measures. We release a simulated dataset, COB-3D, which highlights new types of ambiguity that arise in real-world robotics applications.
  arXiv  Detail & Related papers  (2022-10-13T23:57:40Z)
• Density-aware NeRF Ensembles: Quantifying Predictive Uncertainty in Neural Radiance Fields [7.380217868660371]
  We show that ensembling effectively quantifies model uncertainty in Neural Radiance Fields (NeRFs) We demonstrate that NeRF uncertainty can be utilised for next-best view selection and model refinement.
  arXiv  Detail & Related papers  (2022-09-19T02:28:33Z)
• Localization Uncertainty Estimation for Anchor-Free Object Detection [48.931731695431374]
  There are several limitations of the existing uncertainty estimation methods for anchor-based object detection. We propose a new localization uncertainty estimation method called UAD for anchor-free object detection. Our method captures the uncertainty in four directions of box offsets that are homogeneous, so that it can tell which direction is uncertain.
  arXiv  Detail & Related papers  (2020-06-28T13:49:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.