Continuous Pose for Monocular Cameras in Neural Implicit Representation

• URL: http://arxiv.org/abs/2311.17119v3
• Date: Sat, 2 Mar 2024 13:07:56 GMT
• Title: Continuous Pose for Monocular Cameras in Neural Implicit Representation
• Authors: Qi Ma, Danda Pani Paudel, Ajad Chhatkuli, Luc Van Gool
• Abstract summary: In this paper, we showcase the effectiveness of optimizing monocular camera poses as a continuous function of time. We exploit the proposed method in four diverse experimental settings. Using the assumption of continuous motion, changes in pose may actually live in a manifold that has lower than 6 degrees of freedom (DOF) We call this low DOF motion representation as the emphintrinsic motion and use the approach in vSLAM settings, showing impressive camera tracking performance.
• Score: 65.40527279809474
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we showcase the effectiveness of optimizing monocular camera poses as a continuous function of time. The camera poses are represented using an implicit neural function which maps the given time to the corresponding camera pose. The mapped camera poses are then used for the downstream tasks where joint camera pose optimization is also required. While doing so, the network parameters -- that implicitly represent camera poses -- are optimized. We exploit the proposed method in four diverse experimental settings, namely, (1) NeRF from noisy poses; (2) NeRF from asynchronous Events; (3) Visual Simultaneous Localization and Mapping (vSLAM); and (4) vSLAM with IMUs. In all four settings, the proposed method performs significantly better than the compared baselines and the state-of-the-art methods. Additionally, using the assumption of continuous motion, changes in pose may actually live in a manifold that has lower than 6 degrees of freedom (DOF) is also realized. We call this low DOF motion representation as the \emph{intrinsic motion} and use the approach in vSLAM settings, showing impressive camera tracking performance.

Related papers

• Redundancy-Aware Camera Selection for Indoor Scene Neural Rendering [54.468355408388675]
  We build a similarity matrix that incorporates both the spatial diversity of the cameras and the semantic variation of the images. We apply a diversity-based sampling algorithm to optimize the camera selection. We also develop a new dataset, IndoorTraj, which includes long and complex camera movements captured by humans in virtual indoor environments.
  arXiv  Detail & Related papers  (2024-09-11T08:36:49Z)
• ADen: Adaptive Density Representations for Sparse-view Camera Pose Estimation [17.097170273209333]
  Recovering camera poses from a set of images is a foundational task in 3D computer vision. Recent data-driven approaches aim to directly output camera poses, either through regressing the 6DoF camera poses or formulating rotation as a probability distribution. We propose ADen to unify the two frameworks by employing a generator and a discriminator.
  arXiv  Detail & Related papers  (2024-08-16T22:45:46Z)
• Cameras as Rays: Pose Estimation via Ray Diffusion [54.098613859015856]
  Estimating camera poses is a fundamental task for 3D reconstruction and remains challenging given sparsely sampled views. We propose a distributed representation of camera pose that treats a camera as a bundle of rays. Our proposed methods, both regression- and diffusion-based, demonstrate state-of-the-art performance on camera pose estimation on CO3D.
  arXiv  Detail & Related papers  (2024-02-22T18:59:56Z)
• COLMAP-Free 3D Gaussian Splatting [88.420322646756]
  We propose a novel method to perform novel view synthesis without any SfM preprocessing. We process the input frames in a sequential manner and progressively grow the 3D Gaussians set by taking one input frame at a time. Our method significantly improves over previous approaches in view synthesis and camera pose estimation under large motion changes.
  arXiv  Detail & Related papers  (2023-12-12T18:39:52Z)
• CBARF: Cascaded Bundle-Adjusting Neural Radiance Fields from Imperfect Camera Poses [23.427859480410934]
  We propose a novel 3D reconstruction framework that enables simultaneous optimization of camera poses. In a nutshell, our framework optimize camera poses in a coarse-to-fine manner and then reconstructs scenes based on the rectified poses. Experimental results demonstrate that our CBARF model achieves state-of-the-art performance in both pose optimization and novel view synthesis.
  arXiv  Detail & Related papers  (2023-10-15T08:34:40Z)
• Structure-Aware NeRF without Posed Camera via Epipolar Constraint [8.115535686311249]
  The neural radiance field (NeRF) for realistic novel view synthesis requires camera poses to be pre-acquired. We integrate the pose extraction and view synthesis into a single end-to-end procedure so they can benefit from each other.
  arXiv  Detail & Related papers  (2022-10-01T03:57:39Z)
• ROSEFusion: Random Optimization for Online Dense Reconstruction under Fast Camera Motion [15.873973449155313]
  Reconstruction based on RGB-D sequences has thus far been restrained to relatively slow camera motions (1m/s) Fast motion brings two challenges to depth fusion: 1) the high nonlinearity of camera pose optimization due to large inter-frame rotations and 2) the lack of reliably trackable features due to motion blur. We propose to tackle the difficulties of fast-motion camera tracking in the absence of inertial measurements using random optimization. Thanks to the efficient template-based particle set evolution and the effective fitness function, our method attains good quality pose tracking under fast camera motion (up to 4m/s) in a
  arXiv  Detail & Related papers  (2021-05-12T11:37:34Z)
• Learning Neural Representation of Camera Pose with Matrix Representation of Pose Shift via View Synthesis [105.37072293076767]
  How to effectively represent camera pose is an essential problem in 3D computer vision. We propose an approach to learn neural representations of camera poses and 3D scenes. We conduct extensive experiments on synthetic and real datasets.
  arXiv  Detail & Related papers  (2021-04-04T00:40:53Z)
• GNeRF: GAN-based Neural Radiance Field without Posed Camera [67.80805274569354]
  We introduce GNeRF, a framework to marry Generative Adversarial Networks (GAN) with Neural Radiance Field reconstruction for the complex scenarios with unknown and even randomly camera poses. Our approach outperforms the baselines favorably in those scenes with repeated patterns or even low textures that are regarded as extremely challenging before.
  arXiv  Detail & Related papers  (2021-03-29T13:36:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.