CF-NeRF: Camera Parameter Free Neural Radiance Fields with Incremental Learning

• URL: http://arxiv.org/abs/2312.08760v1
• Date: Thu, 14 Dec 2023 09:09:31 GMT
• Title: CF-NeRF: Camera Parameter Free Neural Radiance Fields with Incremental Learning
• Authors: Qingsong Yan, Qiang Wang, Kaiyong Zhao, Jie Chen, Bo Li, Xiaowen Chu, Fei Deng
• Abstract summary: We propose a novel underlinecamera parameter underlinefree neural radiance field (CF-NeRF) CF-NeRF incrementally reconstructs 3D representations and recovers the camera parameters inspired by incremental structure from motion. Results demonstrate that CF-NeRF is robust to camera rotation and achieves state-of-the-art results without providing prior information and constraints.
• Score: 23.080474939586654
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Neural Radiance Fields (NeRF) have demonstrated impressive performance in novel view synthesis. However, NeRF and most of its variants still rely on traditional complex pipelines to provide extrinsic and intrinsic camera parameters, such as COLMAP. Recent works, like NeRFmm, BARF, and L2G-NeRF, directly treat camera parameters as learnable and estimate them through differential volume rendering. However, these methods work for forward-looking scenes with slight motions and fail to tackle the rotation scenario in practice. To overcome this limitation, we propose a novel \underline{c}amera parameter \underline{f}ree neural radiance field (CF-NeRF), which incrementally reconstructs 3D representations and recovers the camera parameters inspired by incremental structure from motion (SfM). Given a sequence of images, CF-NeRF estimates the camera parameters of images one by one and reconstructs the scene through initialization, implicit localization, and implicit optimization. To evaluate our method, we use a challenging real-world dataset NeRFBuster which provides 12 scenes under complex trajectories. Results demonstrate that CF-NeRF is robust to camera rotation and achieves state-of-the-art results without providing prior information and constraints.

Related papers

• LuSh-NeRF: Lighting up and Sharpening NeRFs for Low-light Scenes [38.59630957057759]
  We propose a novel model, named LuSh-NeRF, which can reconstruct a clean and sharp NeRF from a group of hand-held low-light images. LuSh-NeRF includes a Scene-Noise Decomposition module for decoupling the noise from the scene representation. Experiments show that LuSh-NeRF outperforms existing approaches.
  arXiv  Detail & Related papers  (2024-11-11T07:22:31Z)
• CT-NeRF: Incremental Optimizing Neural Radiance Field and Poses with Complex Trajectory [12.460959809597213]
  We propose CT-NeRF, an incremental reconstruction optimization pipeline using only RGB images without pose and depth input. We evaluate the performance of CT-NeRF on two real-world datasets, NeRFBuster and Free-Dataset.
  arXiv  Detail & Related papers  (2024-04-22T06:07:06Z)
• USB-NeRF: Unrolling Shutter Bundle Adjusted Neural Radiance Fields [7.671858441929298]
  We propose Unrolling Shutter Bundle Adjusted Neural Radiance Fields (USB-NeRF) USB-NeRF is able to correct rolling shutter distortions and recover accurate camera motion trajectory simultaneously under the framework of NeRF. Our algorithm can also be used to recover high-fidelity high frame-rate global shutter video from a sequence of RS images.
  arXiv  Detail & Related papers  (2023-10-04T09:51:58Z)
• Robust e-NeRF: NeRF from Sparse & Noisy Events under Non-Uniform Motion [67.15935067326662]
  Event cameras offer low power, low latency, high temporal resolution and high dynamic range. NeRF is seen as the leading candidate for efficient and effective scene representation. We propose Robust e-NeRF, a novel method to directly and robustly reconstruct NeRFs from moving event cameras.
  arXiv  Detail & Related papers  (2023-09-15T17:52:08Z)
• CamP: Camera Preconditioning for Neural Radiance Fields [56.46526219931002]
  NeRFs can be optimized to obtain high-fidelity 3D scene reconstructions of objects and large-scale scenes. Extrinsic and intrinsic camera parameters are usually estimated using Structure-from-Motion (SfM) methods as a pre-processing step to NeRF. We propose using a proxy problem to compute a whitening transform that eliminates the correlation between camera parameters and normalizes their effects.
  arXiv  Detail & Related papers  (2023-08-21T17:59:54Z)
• NeRFtrinsic Four: An End-To-End Trainable NeRF Jointly Optimizing Diverse Intrinsic and Extrinsic Camera Parameters [7.165373389474194]
  Novel view synthesis using neural radiance fields (NeRF) is the state-of-the-art technique for generating high-quality images from novel viewpoints. Current research on the joint optimization of camera parameters and NeRF focuses on refining noisy extrinsic camera parameters. We propose a novel end-to-end trainable approach called NeRFtrinsic Four to address these limitations.
  arXiv  Detail & Related papers  (2023-03-16T15:44:31Z)
• AligNeRF: High-Fidelity Neural Radiance Fields via Alignment-Aware Training [100.33713282611448]
  We conduct the first pilot study on training NeRF with high-resolution data. We propose the corresponding solutions, including marrying the multilayer perceptron with convolutional layers. Our approach is nearly free without introducing obvious training/testing costs.
  arXiv  Detail & Related papers  (2022-11-17T17:22:28Z)
• CLONeR: Camera-Lidar Fusion for Occupancy Grid-aided Neural Representations [77.90883737693325]
  This paper proposes CLONeR, which significantly improves upon NeRF by allowing it to model large outdoor driving scenes observed from sparse input sensor views. This is achieved by decoupling occupancy and color learning within the NeRF framework into separate Multi-Layer Perceptrons (MLPs) trained using LiDAR and camera data, respectively. In addition, this paper proposes a novel method to build differentiable 3D Occupancy Grid Maps (OGM) alongside the NeRF model, and leverage this occupancy grid for improved sampling of points along a ray for rendering in metric space.
  arXiv  Detail & Related papers  (2022-09-02T17:44:50Z)
• Aug-NeRF: Training Stronger Neural Radiance Fields with Triple-Level Physically-Grounded Augmentations [111.08941206369508]
  We propose Augmented NeRF (Aug-NeRF), which for the first time brings the power of robust data augmentations into regularizing the NeRF training. Our proposal learns to seamlessly blend worst-case perturbations into three distinct levels of the NeRF pipeline. Aug-NeRF effectively boosts NeRF performance in both novel view synthesis and underlying geometry reconstruction.
  arXiv  Detail & Related papers  (2022-07-04T02:27:07Z)
• NeRF--: Neural Radiance Fields Without Known Camera Parameters [31.01560143595185]
  This paper tackles the problem of novel view synthesis (NVS) from 2D images without known camera poses and intrinsics. We propose an end-to-end framework, termed NeRF--, for training NeRF models given only RGB images.
  arXiv  Detail & Related papers  (2021-02-14T03:52:34Z)
• iNeRF: Inverting Neural Radiance Fields for Pose Estimation [68.91325516370013]
  We present iNeRF, a framework that performs mesh-free pose estimation by "inverting" a Neural RadianceField (NeRF) NeRFs have been shown to be remarkably effective for the task of view synthesis.
  arXiv  Detail & Related papers  (2020-12-10T18:36:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.