Instant Uncertainty Calibration of NeRFs Using a Meta-calibrator

• URL: http://arxiv.org/abs/2312.02350v2
• Date: Tue, 19 Mar 2024 21:54:22 GMT
• Title: Instant Uncertainty Calibration of NeRFs Using a Meta-calibrator
• Authors: Niki Amini-Naieni, Tomas Jakab, Andrea Vedaldi, Ronald Clark,
• Abstract summary: We introduce the concept of a meta-calibrator that performs uncertainty calibration for NeRFs with a single forward pass. We show that the meta-calibrator can generalize on unseen scenes and achieves well-calibrated and state-of-the-art uncertainty for NeRFs.
• Score: 60.47106421809998
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Although Neural Radiance Fields (NeRFs) have markedly improved novel view synthesis, accurate uncertainty quantification in their image predictions remains an open problem. The prevailing methods for estimating uncertainty, including the state-of-the-art Density-aware NeRF Ensembles (DANE) [29], quantify uncertainty without calibration. This frequently leads to over- or under-confidence in image predictions, which can undermine their real-world applications. In this paper, we propose a method which, for the first time, achieves calibrated uncertainties for NeRFs. To accomplish this, we overcome a significant challenge in adapting existing calibration techniques to NeRFs: a need to hold out ground truth images from the target scene, reducing the number of images left to train the NeRF. This issue is particularly problematic in sparse-view settings, where we can operate with as few as three images. To address this, we introduce the concept of a meta-calibrator that performs uncertainty calibration for NeRFs with a single forward pass without the need for holding out any images from the target scene. Our meta-calibrator is a neural network that takes as input the NeRF images and uncalibrated uncertainty maps and outputs a scene-specific calibration curve that corrects the NeRF's uncalibrated uncertainties. We show that the meta-calibrator can generalize on unseen scenes and achieves well-calibrated and state-of-the-art uncertainty for NeRFs, significantly beating DANE and other approaches. This opens opportunities to improve applications that rely on accurate NeRF uncertainty estimates such as next-best view planning and potentially more trustworthy image reconstruction for medical diagnosis.

Related papers

• OPONeRF: One-Point-One NeRF for Robust Neural Rendering [70.56874833759241]
  We propose a One-Point-One NeRF (OPONeRF) framework for robust scene rendering. Small but unpredictable perturbations such as object movements, light changes and data contaminations broadly exist in real-life 3D scenes. Experimental results show that our OPONeRF outperforms state-of-the-art NeRFs on various evaluation metrics.
  arXiv  Detail & Related papers  (2024-09-30T07:49:30Z)
• Clean-NeRF: Reformulating NeRF to account for View-Dependent Observations [67.54358911994967]
  This paper proposes Clean-NeRF for accurate 3D reconstruction and novel view rendering in complex scenes. Clean-NeRF can be implemented as a plug-in that can immediately benefit existing NeRF-based methods without additional input.
  arXiv  Detail & Related papers  (2023-03-26T12:24:31Z)
• Self-NeRF: A Self-Training Pipeline for Few-Shot Neural Radiance Fields [17.725937326348994]
  We propose Self-NeRF, a self-evolved NeRF that iteratively refines the radiance fields with very few number of input views. In each iteration, we label unseen views with the predicted colors or warped pixels generated by the model from the preceding iteration. These expanded pseudo-views are afflicted by imprecision in color and warping artifacts, which degrades the performance of NeRF.
  arXiv  Detail & Related papers  (2023-03-10T08:22:36Z)
• Density-aware NeRF Ensembles: Quantifying Predictive Uncertainty in Neural Radiance Fields [7.380217868660371]
  We show that ensembling effectively quantifies model uncertainty in Neural Radiance Fields (NeRFs) We demonstrate that NeRF uncertainty can be utilised for next-best view selection and model refinement.
  arXiv  Detail & Related papers  (2022-09-19T02:28:33Z)
• E-NeRF: Neural Radiance Fields from a Moving Event Camera [83.91656576631031]
  Estimating neural radiance fields (NeRFs) from ideal images has been extensively studied in the computer vision community. We present E-NeRF, the first method which estimates a volumetric scene representation in the form of a NeRF from a fast-moving event camera.
  arXiv  Detail & Related papers  (2022-08-24T04:53:32Z)
• Gradient-based Uncertainty for Monocular Depth Estimation [5.7575052885308455]
  In monocular depth estimation, disturbances in the image context, like moving objects or reflecting materials, can easily lead to erroneous predictions. We propose a post hoc uncertainty estimation approach for an already trained and thus fixed depth estimation model. Our approach achieves state-of-the-art uncertainty estimation results on the KITTI and NYU Depth V2 benchmarks without the need to retrain the neural network.
  arXiv  Detail & Related papers  (2022-08-03T12:21:02Z)
• Conditional-Flow NeRF: Accurate 3D Modelling with Reliable Uncertainty Quantification [44.598503284186336]
  Conditional-Flow NeRF (CF-NeRF) is a novel probabilistic framework to incorporate uncertainty quantification into NeRF-based approaches. CF-NeRF learns a distribution over all possible radiance fields modelling which is used to quantify the uncertainty associated with the modelled scene.
  arXiv  Detail & Related papers  (2022-03-18T23:26:20Z)
• Deep Bayesian inference for seismic imaging with tasks [0.6445605125467573]
  We propose to use techniques from Bayesian inference and deep neural networks to translate uncertainty in seismic imaging to uncertainty in tasks performed on the image. A systematic approach is proposed to translate uncertainty due to noise in the data to confidence intervals of automatically tracked horizons in the image.
  arXiv  Detail & Related papers  (2021-10-10T15:25:44Z)
• Salvage Reusable Samples from Noisy Data for Robust Learning [70.48919625304]
  We propose a reusable sample selection and correction approach, termed as CRSSC, for coping with label noise in training deep FG models with web images. Our key idea is to additionally identify and correct reusable samples, and then leverage them together with clean examples to update the networks.
  arXiv  Detail & Related papers  (2020-08-06T02:07:21Z)
• Calibrating Deep Neural Networks using Focal Loss [77.92765139898906]
  Miscalibration is a mismatch between a model's confidence and its correctness. We show that focal loss allows us to learn models that are already very well calibrated. We show that our approach achieves state-of-the-art calibration without compromising on accuracy in almost all cases.
  arXiv  Detail & Related papers  (2020-02-21T17:35:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.