Photometric LiDAR and RGB-D Bundle Adjustment

• URL: http://arxiv.org/abs/2303.16878v1
• Date: Wed, 29 Mar 2023 17:35:23 GMT
• Title: Photometric LiDAR and RGB-D Bundle Adjustment
• Authors: Luca Di Giammarino and Emanuele Giacomini and Leonardo Brizi and Omar Salem and Giorgio Grisetti
• Abstract summary: This paper presents a novel Bundle Adjustment (BA) photometric strategy that accounts for both RGB-D and LiDAR in the same way. In addition, we present the benefit of jointly using RGB-D and LiDAR within our unified method.
• Score: 3.3948742816399697
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The joint optimization of the sensor trajectory and 3D map is a crucial characteristic of Simultaneous Localization and Mapping (SLAM) systems. To achieve this, the gold standard is Bundle Adjustment (BA). Modern 3D LiDARs now retain higher resolutions that enable the creation of point cloud images resembling those taken by conventional cameras. Nevertheless, the typical effective global refinement techniques employed for RGB-D sensors are not widely applied to LiDARs. This paper presents a novel BA photometric strategy that accounts for both RGB-D and LiDAR in the same way. Our work can be used on top of any SLAM/GNSS estimate to improve and refine the initial trajectory. We conducted different experiments using these two depth sensors on public benchmarks. Our results show that our system performs on par or better compared to other state-of-the-art ad-hoc SLAM/BA strategies, free from data association and without making assumptions about the environment. In addition, we present the benefit of jointly using RGB-D and LiDAR within our unified method. We finally release an open-source CUDA/C++ implementation.

Related papers

• LiOn-XA: Unsupervised Domain Adaptation via LiDAR-Only Cross-Modal Adversarial Training [61.26381389532653]
  LiOn-XA is an unsupervised domain adaptation (UDA) approach that combines LiDAR-Only Cross-Modal (X) learning with Adversarial training for 3D LiDAR point cloud semantic segmentation. Our experiments on 3 real-to-real adaptation scenarios demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-10-21T09:50:17Z)
• Gait Sequence Upsampling using Diffusion Models for Single LiDAR Sensors [1.0485739694839664]
  LidarGSU is designed to improve the generalization capability of existing identification models. In this work, we leverage DPMs on sparse sequential pedestrian point clouds as conditional masks in a video-to-video translation approach. We conduct extensive experiments on the SUSTeck1K dataset to evaluate the generative quality and recognition performance of the proposed method.
  arXiv  Detail & Related papers  (2024-10-11T10:11:21Z)
• IG-SLAM: Instant Gaussian SLAM [6.228980850646457]
  3D Gaussian Splatting has recently shown promising results as an alternative scene representation in SLAM systems. We present IG-SLAM, a dense RGB-only SLAM system that employs robust Dense-SLAM methods for tracking and combines them with Gaussian Splatting. We demonstrate competitive performance with state-of-the-art RGB-only SLAM systems while achieving faster operation speeds.
  arXiv  Detail & Related papers  (2024-08-02T09:07:31Z)
• GlORIE-SLAM: Globally Optimized RGB-only Implicit Encoding Point Cloud SLAM [53.6402869027093]
  We propose an efficient RGB-only dense SLAM system using a flexible neural point cloud representation scene. We also introduce a novel DSPO layer for bundle adjustment which optimize the pose and depth of implicits along with the scale of the monocular depth.
  arXiv  Detail & Related papers  (2024-03-28T16:32:06Z)
• CMRNext: Camera to LiDAR Matching in the Wild for Localization and Extrinsic Calibration [9.693729708337125]
  CMRNext is a novel approach for camera-LIDAR matching that is independent of sensor-specific parameters, generalizable, and can be used in the wild. We extensively evaluate CMRNext on six different robotic platforms, including three publicly available datasets and three in-house robots.
  arXiv  Detail & Related papers  (2024-01-31T19:14:12Z)
• NICER-SLAM: Neural Implicit Scene Encoding for RGB SLAM [111.83168930989503]
  NICER-SLAM is a dense RGB SLAM system that simultaneously optimize for camera poses and a hierarchical neural implicit map representation. We show strong performance in dense mapping, tracking, and novel view synthesis, even competitive with recent RGB-D SLAM systems.
  arXiv  Detail & Related papers  (2023-02-07T17:06:34Z)
• R$^3$LIVE++: A Robust, Real-time, Radiance reconstruction package with a tightly-coupled LiDAR-Inertial-Visual state Estimator [5.972044427549262]
  Simultaneous localization and mapping (SLAM) are crucial for autonomous robots (e.g., self-driving cars, autonomous drones), 3D mapping systems, and AR/VR applications. This work proposed a novel LiDAR-inertial-visual fusion framework termed R$3$LIVE++ to achieve robust and accurate state estimation while simultaneously reconstructing the radiance map on the fly.
  arXiv  Detail & Related papers  (2022-09-08T09:26:20Z)
• Benchmarking the Robustness of LiDAR-Camera Fusion for 3D Object Detection [58.81316192862618]
  Two critical sensors for 3D perception in autonomous driving are the camera and the LiDAR. fusing these two modalities can significantly boost the performance of 3D perception models. We benchmark the state-of-the-art fusion methods for the first time.
  arXiv  Detail & Related papers  (2022-05-30T09:35:37Z)
• LiDARCap: Long-range Marker-less 3D Human Motion Capture with LiDAR Point Clouds [58.402752909624716]
  Existing motion capture datasets are largely short-range and cannot yet fit the need of long-range applications. We propose LiDARHuman26M, a new human motion capture dataset captured by LiDAR at a much longer range to overcome this limitation. Our dataset also includes the ground truth human motions acquired by the IMU system and the synchronous RGB images.
  arXiv  Detail & Related papers  (2022-03-28T12:52:45Z)
• End-To-End Optimization of LiDAR Beam Configuration for 3D Object Detection and Localization [87.56144220508587]
  We take a new route to learn to optimize the LiDAR beam configuration for a given application. We propose a reinforcement learning-based learning-to-optimize framework to automatically optimize the beam configuration. Our method is especially useful when a low-resolution (low-cost) LiDAR is needed.
  arXiv  Detail & Related papers  (2022-01-11T09:46:31Z)
• DeepLiDARFlow: A Deep Learning Architecture For Scene Flow Estimation Using Monocular Camera and Sparse LiDAR [10.303618438296981]
  Scene flow is the dense 3D reconstruction of motion and geometry of a scene. Most state-of-the-art methods use a pair of stereo images as input for full scene reconstruction. DeepLiDARFlow is a novel deep learning architecture which fuses high level RGB and LiDAR features at multiple scales.
  arXiv  Detail & Related papers  (2020-08-18T19:51:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.