NeRF and Gaussian Splatting SLAM in the Wild

• URL: http://arxiv.org/abs/2412.03263v1
• Date: Wed, 04 Dec 2024 12:11:19 GMT
• Title: NeRF and Gaussian Splatting SLAM in the Wild
• Authors: Fabian Schmidt, Markus Enzweiler, Abhinav Valada,
• Abstract summary: This study focuses on camera tracking accuracy, robustness to environmental factors, and computational efficiency, highlighting distinct trade-offs. Neural SLAM methods achieve superior robustness, particularly under challenging conditions such as low light, but at a high computational cost. Traditional methods perform the best across seasons but are highly sensitive to variations in lighting conditions.
• Score: 9.516289996766059
• License:
• Abstract: Navigating outdoor environments with visual Simultaneous Localization and Mapping (SLAM) systems poses significant challenges due to dynamic scenes, lighting variations, and seasonal changes, requiring robust solutions. While traditional SLAM methods struggle with adaptability, deep learning-based approaches and emerging neural radiance fields as well as Gaussian Splatting-based SLAM methods, offer promising alternatives. However, these methods have primarily been evaluated in controlled indoor environments with stable conditions, leaving a gap in understanding their performance in unstructured and variable outdoor settings. This study addresses this gap by evaluating these methods in natural outdoor environments, focusing on camera tracking accuracy, robustness to environmental factors, and computational efficiency, highlighting distinct trade-offs. Extensive evaluations demonstrate that neural SLAM methods achieve superior robustness, particularly under challenging conditions such as low light, but at a high computational cost. At the same time, traditional methods perform the best across seasons but are highly sensitive to variations in lighting conditions. The code of the benchmark is publicly available at https://github.com/iis-esslingen/nerf-3dgs-benchmark.

Related papers

• ROVER: A Multi-Season Dataset for Visual SLAM [8.711135744156564]
  ROVER is a benchmark dataset tailored for evaluating visual SLAM algorithms under diverse environmental conditions. It covers 39 recordings across five outdoor locations, collected through all seasons and various lighting scenarios. Results demonstrate that while stereo-inertial and RGB-D configurations generally perform better under favorable lighting and moderate vegetation, most SLAM systems perform poorly in low-light and high-vegetation scenarios.
  arXiv  Detail & Related papers  (2024-12-03T15:34:00Z)
• DeepArUco++: Improved detection of square fiducial markers in challenging lighting conditions [3.783609886054562]
  Fiducial markers are a computer vision tool used for object pose estimation and detection. DeepArUco++ is a framework that performs marker detection and decoding in challenging lighting conditions. We present a second, real-life dataset of ArUco markers in challenging lighting conditions used to evaluate our system.
  arXiv  Detail & Related papers  (2024-11-08T13:18:31Z)
• SLAIM: Robust Dense Neural SLAM for Online Tracking and Mapping [15.63276368052395]
  We propose a novel coarse-to-fine tracking model tailored for Neural Radiance Field SLAM (NeRF-SLAM) Existing NeRF-SLAM systems consistently exhibit inferior tracking performance compared to traditional SLAM algorithms. We implement both local and global bundle-adjustment to produce a robust (coarse-to-fine) and accurate (KL regularizer) SLAM solution.
  arXiv  Detail & Related papers  (2024-04-17T14:23:28Z)
• DK-SLAM: Monocular Visual SLAM with Deep Keypoint Learning, Tracking and Loop-Closing [13.50980509878613]
  Experimental evaluations on publicly available datasets demonstrate that DK-SLAM outperforms leading traditional and learning based SLAM systems. Our system employs a Model-Agnostic Meta-Learning (MAML) strategy to optimize the training of keypoint extraction networks. To mitigate cumulative positioning errors, DK-SLAM incorporates a novel online learning module that utilizes binary features for loop closure detection.
  arXiv  Detail & Related papers  (2024-01-17T12:08:30Z)
• Implicit Event-RGBD Neural SLAM [54.74363487009845]
  Implicit neural SLAM has achieved remarkable progress recently. Existing methods face significant challenges in non-ideal scenarios. We propose EN-SLAM, the first event-RGBD implicit neural SLAM framework.
  arXiv  Detail & Related papers  (2023-11-18T08:48:58Z)
• Toward Fast, Flexible, and Robust Low-Light Image Enhancement [87.27326390675155]
  We develop a new Self-Calibrated Illumination (SCI) learning framework for fast, flexible, and robust brightening images in real-world low-light scenarios. Considering the computational burden of the cascaded pattern, we construct the self-calibrated module which realizes the convergence between results of each stage. We make comprehensive explorations to SCI's inherent properties including operation-insensitive adaptability and model-irrelevant generality.
  arXiv  Detail & Related papers  (2022-04-21T14:40:32Z)
• Learning to Continuously Optimize Wireless Resource in a Dynamic Environment: A Bilevel Optimization Perspective [52.497514255040514]
  This work develops a new approach that enables data-driven methods to continuously learn and optimize resource allocation strategies in a dynamic environment. We propose to build the notion of continual learning into wireless system design, so that the learning model can incrementally adapt to the new episodes. Our design is based on a novel bilevel optimization formulation which ensures certain fairness" across different data samples.
  arXiv  Detail & Related papers  (2021-05-03T07:23:39Z)
• LIFT-SLAM: a deep-learning feature-based monocular visual SLAM method [0.0]
  We propose to combine the potential of deep learning-based feature descriptors with the traditional geometry-based VSLAM. Experiments conducted on KITTI and Euroc datasets show that deep learning can be used to improve the performance of traditional VSLAM systems.
  arXiv  Detail & Related papers  (2021-03-31T20:35:10Z)
• Learning to Continuously Optimize Wireless Resource In Episodically Dynamic Environment [55.91291559442884]
  This work develops a methodology that enables data-driven methods to continuously learn and optimize in a dynamic environment. We propose to build the notion of continual learning into the modeling process of learning wireless systems. Our design is based on a novel min-max formulation which ensures certain fairness" across different data samples.
  arXiv  Detail & Related papers  (2020-11-16T08:24:34Z)
• Pushing the Envelope of Rotation Averaging for Visual SLAM [69.7375052440794]
  We propose a novel optimization backbone for visual SLAM systems. We leverage averaging to improve the accuracy, efficiency and robustness of conventional monocular SLAM systems. Our approach can exhibit up to 10x faster with comparable accuracy against the state-art on public benchmarks.
  arXiv  Detail & Related papers  (2020-11-02T18:02:26Z)
• Object-based Illumination Estimation with Rendering-aware Neural Networks [56.01734918693844]
  We present a scheme for fast environment light estimation from the RGBD appearance of individual objects and their local image areas. With the estimated lighting, virtual objects can be rendered in AR scenarios with shading that is consistent to the real scene.
  arXiv  Detail & Related papers  (2020-08-06T08:23:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.