Ternary-Type Opacity and Hybrid Odometry for RGB NeRF-SLAM

• URL: http://arxiv.org/abs/2312.13332v3
• Date: Mon, 23 Sep 2024 14:48:06 GMT
• Title: Ternary-Type Opacity and Hybrid Odometry for RGB NeRF-SLAM
• Authors: Junru Lin, Asen Nachkov, Songyou Peng, Luc Van Gool, Danda Pani Paudel,
• Abstract summary: We introduce a ternary-type opacity (TT) model, which categorizes points on a ray intersecting a surface into three regions: before, on, and behind the surface. This enables a more accurate rendering of depth, subsequently improving the performance of image warping techniques. Our integrated approach of TT and HO achieves state-of-the-art performance on synthetic and real-world datasets.
• Score: 58.736472371951955
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this work, we address the challenge of deploying Neural Radiance Field (NeRFs) in Simultaneous Localization and Mapping (SLAM) under the condition of lacking depth information, relying solely on RGB inputs. The key to unlocking the full potential of NeRF in such a challenging context lies in the integration of real-world priors. A crucial prior we explore is the binary opacity prior of 3D space with opaque objects. To effectively incorporate this prior into the NeRF framework, we introduce a ternary-type opacity (TT) model instead, which categorizes points on a ray intersecting a surface into three regions: before, on, and behind the surface. This enables a more accurate rendering of depth, subsequently improving the performance of image warping techniques. Therefore, we further propose a novel hybrid odometry (HO) scheme that merges bundle adjustment and warping-based localization. Our integrated approach of TT and HO achieves state-of-the-art performance on synthetic and real-world datasets, in terms of both speed and accuracy. This breakthrough underscores the potential of NeRF-SLAM in navigating complex environments with high fidelity.

Related papers

• RaNeuS: Ray-adaptive Neural Surface Reconstruction [87.20343320266215]
  We leverage a differentiable radiance field eg NeRF to reconstruct detailed 3D surfaces in addition to producing novel view renderings. Considering that different methods formulate and optimize the projection from SDF to radiance field with a globally constant Eikonal regularization, we improve with a ray-wise weighting factor. Our proposed textitRaNeuS are extensively evaluated on both synthetic and real datasets.
  arXiv  Detail & Related papers  (2024-06-14T07:54:25Z)
• GlORIE-SLAM: Globally Optimized RGB-only Implicit Encoding Point Cloud SLAM [53.6402869027093]
  We propose an efficient RGB-only dense SLAM system using a flexible neural point cloud representation scene. We also introduce a novel DSPO layer for bundle adjustment which optimize the pose and depth of implicits along with the scale of the monocular depth.
  arXiv  Detail & Related papers  (2024-03-28T16:32:06Z)
• CVT-xRF: Contrastive In-Voxel Transformer for 3D Consistent Radiance Fields from Sparse Inputs [65.80187860906115]
  We propose a novel approach to improve NeRF's performance with sparse inputs. We first adopt a voxel-based ray sampling strategy to ensure that the sampled rays intersect with a certain voxel in 3D space. We then randomly sample additional points within the voxel and apply a Transformer to infer the properties of other points on each ray, which are then incorporated into the volume rendering.
  arXiv  Detail & Related papers  (2024-03-25T15:56:17Z)
• Hi-Map: Hierarchical Factorized Radiance Field for High-Fidelity Monocular Dense Mapping [51.739466714312805]
  We introduce Hi-Map, a novel monocular dense mapping approach based on Neural Radiance Field (NeRF) Hi-Map is exceptional in its capacity to achieve efficient and high-fidelity mapping using only posed RGB inputs.
  arXiv  Detail & Related papers  (2024-01-06T12:32:25Z)
• HI-SLAM: Monocular Real-time Dense Mapping with Hybrid Implicit Fields [11.627951040865568]
  Recent neural mapping frameworks show promising results, but rely on RGB-D or pose inputs, or cannot run in real-time. Our approach integrates dense-SLAM with neural implicit fields. For efficient construction of neural fields, we employ multi-resolution grid encoding and signed distance function.
  arXiv  Detail & Related papers  (2023-10-07T12:26:56Z)
• DaRF: Boosting Radiance Fields from Sparse Inputs with Monocular Depth Adaptation [31.655818586634258]
  We propose a novel framework, dubbed D"aRF, that achieves robust NeRF reconstruction with a handful of real-world images. Our framework imposes the MDE network's powerful geometry prior to NeRF representation at both seen and unseen viewpoints. In addition, we overcome the ambiguity problems of monocular depths through patch-wise scale-shift fitting and geometry distillation.
  arXiv  Detail & Related papers  (2023-05-30T16:46:41Z)
• Learning Neural Duplex Radiance Fields for Real-Time View Synthesis [33.54507228895688]
  We propose a novel approach to distill and bake NeRFs into highly efficient mesh-based neural representations. We demonstrate the effectiveness and superiority of our approach via extensive experiments on a range of standard datasets.
  arXiv  Detail & Related papers  (2023-04-20T17:59:52Z)
• NICER-SLAM: Neural Implicit Scene Encoding for RGB SLAM [111.83168930989503]
  NICER-SLAM is a dense RGB SLAM system that simultaneously optimize for camera poses and a hierarchical neural implicit map representation. We show strong performance in dense mapping, tracking, and novel view synthesis, even competitive with recent RGB-D SLAM systems.
  arXiv  Detail & Related papers  (2023-02-07T17:06:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.