PoRF: Pose Residual Field for Accurate Neural Surface Reconstruction
- URL: http://arxiv.org/abs/2310.07449v3
- Date: Tue, 12 Mar 2024 06:39:02 GMT
- Title: PoRF: Pose Residual Field for Accurate Neural Surface Reconstruction
- Authors: Jia-Wang Bian, Wenjing Bian, Victor Adrian Prisacariu, Philip Torr
- Abstract summary: We introduce the pose residual field (PoRF), a novel implicit representation that uses an correspondence for regressing pose updates.
On the DTU dataset, we reduce the rotation error by 78% for COLMAP poses, leading to the decreased reconstruction Chamfer distance from 3.48mm to 0.85mm.
On the Mobile dataset that contains casually captured 360-degree videos, our method refines ARKit poses and improves the reconstruction F1 score from 69.18 to 75.67.
- Score: 24.882446212756626
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Neural surface reconstruction is sensitive to the camera pose noise, even if
state-of-the-art pose estimators like COLMAP or ARKit are used. More
importantly, existing Pose-NeRF joint optimisation methods have struggled to
improve pose accuracy in challenging real-world scenarios. To overcome the
challenges, we introduce the pose residual field (PoRF), a novel implicit
representation that uses an MLP for regressing pose updates. This is more
robust than the conventional pose parameter optimisation due to parameter
sharing that leverages global information over the entire sequence.
Furthermore, we propose an epipolar geometry loss to enhance the supervision
that leverages the correspondences exported from COLMAP results without the
extra computational overhead. Our method yields promising results. On the DTU
dataset, we reduce the rotation error by 78\% for COLMAP poses, leading to the
decreased reconstruction Chamfer distance from 3.48mm to 0.85mm. On the
MobileBrick dataset that contains casually captured unbounded 360-degree
videos, our method refines ARKit poses and improves the reconstruction F1 score
from 69.18 to 75.67, outperforming that with the dataset provided ground-truth
pose (75.14). These achievements demonstrate the efficacy of our approach in
refining camera poses and improving the accuracy of neural surface
reconstruction in real-world scenarios.
Related papers
- Robust SG-NeRF: Robust Scene Graph Aided Neural Surface Reconstruction [9.429618155943635]
Existing pose-NeRF joint optimization methods handle poses with small noise (inliers) effectively but struggle with large noise (outliers)
In this work, we focus on mitigating the impact of outlier poses.
Our method integrates an inlier-outlier confidence estimation scheme, leveraging scene graph information gathered during the data preparation phase.
arXiv Detail & Related papers (2024-11-20T09:48:06Z) - SG-NeRF: Neural Surface Reconstruction with Scene Graph Optimization [16.460851701725392]
We present a novel approach that optimize radiance fields with scene graphs to mitigate the influence of outlier poses.
Our method incorporates an adaptive inlier-outlier confidence estimation scheme based on scene graphs.
We also introduce an effective intersection-over-union (IoU) loss to optimize the camera pose and surface geometry.
arXiv Detail & Related papers (2024-07-17T15:50:17Z) - Cameras as Rays: Pose Estimation via Ray Diffusion [54.098613859015856]
Estimating camera poses is a fundamental task for 3D reconstruction and remains challenging given sparsely sampled views.
We propose a distributed representation of camera pose that treats a camera as a bundle of rays.
Our proposed methods, both regression- and diffusion-based, demonstrate state-of-the-art performance on camera pose estimation on CO3D.
arXiv Detail & Related papers (2024-02-22T18:59:56Z) - NoPose-NeuS: Jointly Optimizing Camera Poses with Neural Implicit
Surfaces for Multi-view Reconstruction [0.0]
NoPose-NeuS is a neural implicit surface reconstruction method that extends NeuS to jointly optimize camera poses with the geometry and color networks.
We show that the proposed method can estimate relatively accurate camera poses, while maintaining a high surface reconstruction quality with 0.89 mean Chamfer distance.
arXiv Detail & Related papers (2023-12-23T12:18:22Z) - Indoor Scene Reconstruction with Fine-Grained Details Using Hybrid Representation and Normal Prior Enhancement [50.56517624931987]
The reconstruction of indoor scenes from multi-view RGB images is challenging due to the coexistence of flat and texture-less regions.
Recent methods leverage neural radiance fields aided by predicted surface normal priors to recover the scene geometry.
This work aims to reconstruct high-fidelity surfaces with fine-grained details by addressing the above limitations.
arXiv Detail & Related papers (2023-09-14T12:05:29Z) - LoLep: Single-View View Synthesis with Locally-Learned Planes and
Self-Attention Occlusion Inference [66.45326873274908]
We propose a novel method, LoLep, which regresses Locally-Learned planes from a single RGB image to represent scenes accurately.
Compared to MINE, our approach has an LPIPS reduction of 4.8%-9.0% and an RV reduction of 73.9%-83.5%.
arXiv Detail & Related papers (2023-07-23T03:38:55Z) - LU-NeRF: Scene and Pose Estimation by Synchronizing Local Unposed NeRFs [56.050550636941836]
A critical obstacle preventing NeRF models from being deployed broadly in the wild is their reliance on accurate camera poses.
We propose a novel approach, LU-NeRF, that jointly estimates camera poses and neural fields with relaxed assumptions on pose configuration.
We show our LU-NeRF pipeline outperforms prior attempts at unposed NeRF without making restrictive assumptions on the pose prior.
arXiv Detail & Related papers (2023-06-08T17:56:22Z) - RNNPose: Recurrent 6-DoF Object Pose Refinement with Robust
Correspondence Field Estimation and Pose Optimization [46.144194562841435]
We propose a framework based on a recurrent neural network (RNN) for object pose refinement.
The problem is formulated as a non-linear least squares problem based on the estimated correspondence field.
The correspondence field estimation and pose refinement are conducted alternatively in each iteration to recover accurate object poses.
arXiv Detail & Related papers (2022-03-24T06:24:55Z) - LENS: Localization enhanced by NeRF synthesis [3.4386226615580107]
We demonstrate improvement of camera pose regression thanks to an additional synthetic dataset rendered by the NeRF class of algorithm.
We further improved localization accuracy of pose regressors using synthesized realistic and geometry consistent images as data augmentation during training.
arXiv Detail & Related papers (2021-10-13T08:15:08Z) - FasterPose: A Faster Simple Baseline for Human Pose Estimation [65.8413964785972]
We propose a design paradigm for cost-effective network with LR representation for efficient pose estimation, named FasterPose.
We study the training behavior of FasterPose, and formulate a novel regressive cross-entropy (RCE) loss function for accelerating the convergence.
Compared with the previously dominant network of pose estimation, our method reduces 58% of the FLOPs and simultaneously gains 1.3% improvement of accuracy.
arXiv Detail & Related papers (2021-07-07T13:39:08Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.