CT-NeRF: Incremental Optimizing Neural Radiance Field and Poses with Complex Trajectory

• URL: http://arxiv.org/abs/2404.13896v2
• Date: Tue, 23 Apr 2024 13:02:37 GMT
• Title: CT-NeRF: Incremental Optimizing Neural Radiance Field and Poses with Complex Trajectory
• Authors: Yunlong Ran, Yanxu Li, Qi Ye, Yuchi Huo, Zechun Bai, Jiahao Sun, Jiming Chen,
• Abstract summary: We propose CT-NeRF, an incremental reconstruction optimization pipeline using only RGB images without pose and depth input. We evaluate the performance of CT-NeRF on two real-world datasets, NeRFBuster and Free-Dataset.
• Score: 12.460959809597213
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neural radiance field (NeRF) has achieved impressive results in high-quality 3D scene reconstruction. However, NeRF heavily relies on precise camera poses. While recent works like BARF have introduced camera pose optimization within NeRF, their applicability is limited to simple trajectory scenes. Existing methods struggle while tackling complex trajectories involving large rotations. To address this limitation, we propose CT-NeRF, an incremental reconstruction optimization pipeline using only RGB images without pose and depth input. In this pipeline, we first propose a local-global bundle adjustment under a pose graph connecting neighboring frames to enforce the consistency between poses to escape the local minima caused by only pose consistency with the scene structure. Further, we instantiate the consistency between poses as a reprojected geometric image distance constraint resulting from pixel-level correspondences between input image pairs. Through the incremental reconstruction, CT-NeRF enables the recovery of both camera poses and scene structure and is capable of handling scenes with complex trajectories. We evaluate the performance of CT-NeRF on two real-world datasets, NeRFBuster and Free-Dataset, which feature complex trajectories. Results show CT-NeRF outperforms existing methods in novel view synthesis and pose estimation accuracy.

Related papers

• Few-shot NeRF by Adaptive Rendering Loss Regularization [78.50710219013301]
  Novel view synthesis with sparse inputs poses great challenges to Neural Radiance Field (NeRF) Recent works demonstrate that the frequency regularization of Positional rendering can achieve promising results for few-shot NeRF. We propose Adaptive Rendering loss regularization for few-shot NeRF, dubbed AR-NeRF.
  arXiv  Detail & Related papers  (2024-10-23T13:05:26Z)
• Blending Distributed NeRFs with Tri-stage Robust Pose Optimization [14.36438070050125]
  We present a distributed NeRF system with tri-stage pose optimization. In this paper, we present a distributed NeRF system with tri-stage pose optimization.
  arXiv  Detail & Related papers  (2024-05-05T10:27:03Z)
• CF-NeRF: Camera Parameter Free Neural Radiance Fields with Incremental Learning [23.080474939586654]
  We propose a novel underlinecamera parameter underlinefree neural radiance field (CF-NeRF) CF-NeRF incrementally reconstructs 3D representations and recovers the camera parameters inspired by incremental structure from motion. Results demonstrate that CF-NeRF is robust to camera rotation and achieves state-of-the-art results without providing prior information and constraints.
  arXiv  Detail & Related papers  (2023-12-14T09:09:31Z)
• Pose-Free Neural Radiance Fields via Implicit Pose Regularization [117.648238941948]
  IR-NeRF is an innovative pose-free neural radiance field (NeRF) that introduces implicit pose regularization to refine pose estimator with unposed real images. With a collection of 2D images of a specific scene, IR-NeRF constructs a scene codebook that stores scene features and captures the scene-specific pose distribution implicitly as priors.
  arXiv  Detail & Related papers  (2023-08-29T06:14:06Z)
• LU-NeRF: Scene and Pose Estimation by Synchronizing Local Unposed NeRFs [56.050550636941836]
  A critical obstacle preventing NeRF models from being deployed broadly in the wild is their reliance on accurate camera poses. We propose a novel approach, LU-NeRF, that jointly estimates camera poses and neural fields with relaxed assumptions on pose configuration. We show our LU-NeRF pipeline outperforms prior attempts at unposed NeRF without making restrictive assumptions on the pose prior.
  arXiv  Detail & Related papers  (2023-06-08T17:56:22Z)
• SPARF: Neural Radiance Fields from Sparse and Noisy Poses [58.528358231885846]
  We introduce Sparse Pose Adjusting Radiance Field (SPARF) to address the challenge of novel-view synthesis. Our approach exploits multi-view geometry constraints in order to jointly learn the NeRF and refine the camera poses.
  arXiv  Detail & Related papers  (2022-11-21T18:57:47Z)
• Aug-NeRF: Training Stronger Neural Radiance Fields with Triple-Level Physically-Grounded Augmentations [111.08941206369508]
  We propose Augmented NeRF (Aug-NeRF), which for the first time brings the power of robust data augmentations into regularizing the NeRF training. Our proposal learns to seamlessly blend worst-case perturbations into three distinct levels of the NeRF pipeline. Aug-NeRF effectively boosts NeRF performance in both novel view synthesis and underlying geometry reconstruction.
  arXiv  Detail & Related papers  (2022-07-04T02:27:07Z)
• iNeRF: Inverting Neural Radiance Fields for Pose Estimation [68.91325516370013]
  We present iNeRF, a framework that performs mesh-free pose estimation by "inverting" a Neural RadianceField (NeRF) NeRFs have been shown to be remarkably effective for the task of view synthesis.
  arXiv  Detail & Related papers  (2020-12-10T18:36:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.