(LC)$^2$: LiDAR-Camera Loop Constraints For Cross-Modal Place Recognition

• URL: http://arxiv.org/abs/2304.08660v1
• Date: Mon, 17 Apr 2023 23:20:16 GMT
• Title: (LC)$^2$: LiDAR-Camera Loop Constraints For Cross-Modal Place Recognition
• Authors: Alex Junho Lee, Seungwon Song, Hyungtae Lim, Woojoo Lee and Hyun Myung
• Abstract summary: We propose a novel cross-matching method, called (LC)$2$, for achieving LiDAR localization without a prior point cloud map. Network is trained to extract localization descriptors from disparity and range images. We demonstrate that LiDAR-based navigation systems could be optimized from image databases and vice versa.
• Score: 0.9449650062296824
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Localization has been a challenging task for autonomous navigation. A loop detection algorithm must overcome environmental changes for the place recognition and re-localization of robots. Therefore, deep learning has been extensively studied for the consistent transformation of measurements into localization descriptors. Street view images are easily accessible; however, images are vulnerable to appearance changes. LiDAR can robustly provide precise structural information. However, constructing a point cloud database is expensive, and point clouds exist only in limited places. Different from previous works that train networks to produce shared embedding directly between the 2D image and 3D point cloud, we transform both data into 2.5D depth images for matching. In this work, we propose a novel cross-matching method, called (LC)$^2$, for achieving LiDAR localization without a prior point cloud map. To this end, LiDAR measurements are expressed in the form of range images before matching them to reduce the modality discrepancy. Subsequently, the network is trained to extract localization descriptors from disparity and range images. Next, the best matches are employed as a loop factor in a pose graph. Using public datasets that include multiple sessions in significantly different lighting conditions, we demonstrated that LiDAR-based navigation systems could be optimized from image databases and vice versa.

Related papers

• LiOn-XA: Unsupervised Domain Adaptation via LiDAR-Only Cross-Modal Adversarial Training [61.26381389532653]
  LiOn-XA is an unsupervised domain adaptation (UDA) approach that combines LiDAR-Only Cross-Modal (X) learning with Adversarial training for 3D LiDAR point cloud semantic segmentation. Our experiments on 3 real-to-real adaptation scenarios demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-10-21T09:50:17Z)
• Monocular Visual Place Recognition in LiDAR Maps via Cross-Modal State Space Model and Multi-View Matching [2.400446821380503]
  We introduce an efficient framework to learn descriptors for both RGB images and point clouds. It takes visual state space model (VMamba) as the backbone and employs a pixel-view-scene joint training strategy. A visible 3D points overlap strategy is then designed to quantify the similarity between point cloud views and RGB images for multi-view supervision.
  arXiv  Detail & Related papers  (2024-10-08T18:31:41Z)
• Self-supervised Learning of LiDAR 3D Point Clouds via 2D-3D Neural Calibration [107.61458720202984]
  This paper introduces a novel self-supervised learning framework for enhancing 3D perception in autonomous driving scenes. We propose the learnable transformation alignment to bridge the domain gap between image and point cloud data. We establish dense 2D-3D correspondences to estimate the rigid pose.
  arXiv  Detail & Related papers  (2024-01-23T02:41:06Z)
• TULIP: Transformer for Upsampling of LiDAR Point Clouds [32.77657816997911]
  LiDAR Up is a challenging task for the perception systems of robots and autonomous vehicles. Recent works propose to solve this problem by converting LiDAR data from 3D Euclidean space into an image super-resolution problem in 2D image space. We propose T geometries, a new method to reconstruct high-resolution LiDAR point clouds from low-resolution LiDAR input.
  arXiv  Detail & Related papers  (2023-12-11T10:43:28Z)
• PolarMix: A General Data Augmentation Technique for LiDAR Point Clouds [100.03877236181546]
  PolarMix is a point cloud augmentation technique that is simple and generic. It can work as plug-and-play for various 3D deep architectures and also performs well for unsupervised domain adaptation.
  arXiv  Detail & Related papers  (2022-07-30T13:52:19Z)
• DeepI2P: Image-to-Point Cloud Registration via Deep Classification [71.3121124994105]
  DeepI2P is a novel approach for cross-modality registration between an image and a point cloud. Our method estimates the relative rigid transformation between the coordinate frames of the camera and Lidar. We circumvent the difficulty by converting the registration problem into a classification and inverse camera projection optimization problem.
  arXiv  Detail & Related papers  (2021-04-08T04:27:32Z)
• City-scale Scene Change Detection using Point Clouds [71.73273007900717]
  We propose a method for detecting structural changes in a city using images captured from mounted cameras over two different times. A direct comparison of the two point clouds for change detection is not ideal due to inaccurate geo-location information. To circumvent this problem, we propose a deep learning-based non-rigid registration on the point clouds. Experiments show that our method is able to detect scene changes effectively, even in the presence of viewpoint and illumination differences.
  arXiv  Detail & Related papers  (2021-03-26T08:04:13Z)
• Robust Place Recognition using an Imaging Lidar [45.37172889338924]
  We propose a methodology for robust, real-time place recognition using an imaging lidar. Our method is truly-invariant and can tackle reverse revisiting and upside-down revisiting.
  arXiv  Detail & Related papers  (2021-03-03T01:08:31Z)
• A Simple and Efficient Registration of 3D Point Cloud and Image Data for Indoor Mobile Mapping System [18.644879251473647]
  registration of 3D LiDAR point clouds with optical images is critical in the combination of multi-source data. Geometric misalignment originally exists in the pose data between LiDAR point clouds and optical images. We develop a simple but efficient registration method to improve the accuracy of the initial pose.
  arXiv  Detail & Related papers  (2020-10-27T13:01:54Z)
• SqueezeSegV3: Spatially-Adaptive Convolution for Efficient Point-Cloud Segmentation [66.49351944322835]
  For large-scale point cloud segmentation, the textitde facto method is to project a 3D point cloud to get a 2D LiDAR image and use convolutions to process it. We propose Spatially-Adaptive Convolution (SAC) to adopt different filters for different locations according to the input image. SAC can be computed efficiently since it can be implemented as a series of element-wise multiplications, im2col, and standard convolution.
  arXiv  Detail & Related papers  (2020-04-03T22:47:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.