Stereo-LiDAR Fusion by Semi-Global Matching With Discrete Disparity-Matching Cost and Semidensification

• URL: http://arxiv.org/abs/2504.05148v1
• Date: Mon, 07 Apr 2025 14:54:08 GMT
• Title: Stereo-LiDAR Fusion by Semi-Global Matching With Discrete Disparity-Matching Cost and Semidensification
• Authors: Yasuhiro Yao, Ryoichi Ishikawa, Takeshi Oishi,
• Abstract summary: We present a real-time, non-learning depth estimation method that fuses Light Detection and Ranging (LiDAR) data with stereo camera input.<n>When it was evaluated on the KITTI dataset, the proposed method achieved an error rate of 2.79%.<n>We believe that the real-time and non-learning nature of our method makes it highly practical for applications in robotics and automation.
• Score: 0.358439716487063
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a real-time, non-learning depth estimation method that fuses Light Detection and Ranging (LiDAR) data with stereo camera input. Our approach comprises three key techniques: Semi-Global Matching (SGM) stereo with Discrete Disparity-matching Cost (DDC), semidensification of LiDAR disparity, and a consistency check that combines stereo images and LiDAR data. Each of these components is designed for parallelization on a GPU to realize real-time performance. When it was evaluated on the KITTI dataset, the proposed method achieved an error rate of 2.79\%, outperforming the previous state-of-the-art real-time stereo-LiDAR fusion method, which had an error rate of 3.05\%. Furthermore, we tested the proposed method in various scenarios, including different LiDAR point densities, varying weather conditions, and indoor environments, to demonstrate its high adaptability. We believe that the real-time and non-learning nature of our method makes it highly practical for applications in robotics and automation.

Related papers

• VaLID: Verification as Late Integration of Detections for LiDAR-Camera Fusion [2.503388496100123]
  Vehicle object detection benefits from both LiDAR and camera data.<n>We propose a model-adaptive late-fusion method, VaLID, which validates whether each predicted bounding box is acceptable.<n>Our approach is model-adaptive and demonstrates state-of-the-art competitive performance even when using generic camera detectors.
  arXiv  Detail & Related papers  (2024-09-23T20:27:10Z)
• Stereo-LiDAR Depth Estimation with Deformable Propagation and Learned Disparity-Depth Conversion [16.164300644900404]
  We propose a novel stereo-LiDAR depth estimation network with Semi-Dense hint Guidance, named SDG-Depth. Our network includes a deformable propagation module for generating a semi-dense hint map and a confidence map by propagating sparse hints using a learned deformable window. Our method is both accurate and efficient. The experimental results on benchmark tests show its superior performance.
  arXiv  Detail & Related papers  (2024-04-11T08:12:48Z)
• Uncertainty Guided Adaptive Warping for Robust and Efficient Stereo Matching [77.133400999703]
  Correlation based stereo matching has achieved outstanding performance. Current methods with a fixed model do not work uniformly well across various datasets. This paper proposes a new perspective to dynamically calculate correlation for robust stereo matching.
  arXiv  Detail & Related papers  (2023-07-26T09:47:37Z)
• MV-JAR: Masked Voxel Jigsaw and Reconstruction for LiDAR-Based Self-Supervised Pre-Training [58.07391711548269]
  Masked Voxel Jigsaw and Reconstruction (MV-JAR) method for LiDAR-based self-supervised pre-training. Masked Voxel Jigsaw and Reconstruction (MV-JAR) method for LiDAR-based self-supervised pre-training.
  arXiv  Detail & Related papers  (2023-03-23T17:59:02Z)
• Resolving Class Imbalance for LiDAR-based Object Detector by Dynamic Weight Average and Contextual Ground Truth Sampling [7.096611243139798]
  Real-world driving datasets often suffer from the problem of data imbalance. We propose a method to address this data imbalance problem. Our experiment with KITTI and nuScenes datasets confirms our proposed method's effectiveness.
  arXiv  Detail & Related papers  (2022-10-07T05:23:25Z)
• Robust and accurate depth estimation by fusing LiDAR and Stereo [8.85338187686374]
  We propose a precision and robust method for fusing the LiDAR and stereo cameras. This method fully combines the advantages of the LiDAR and stereo camera. We evaluate the proposed pipeline on the KITTI benchmark.
  arXiv  Detail & Related papers  (2022-07-13T11:55:15Z)
• Benchmarking the Robustness of LiDAR-Camera Fusion for 3D Object Detection [58.81316192862618]
  Two critical sensors for 3D perception in autonomous driving are the camera and the LiDAR. fusing these two modalities can significantly boost the performance of 3D perception models. We benchmark the state-of-the-art fusion methods for the first time.
  arXiv  Detail & Related papers  (2022-05-30T09:35:37Z)
• LIF-Seg: LiDAR and Camera Image Fusion for 3D LiDAR Semantic Segmentation [78.74202673902303]
  We propose a coarse-tofine LiDAR and camera fusion-based network (termed as LIF-Seg) for LiDAR segmentation. The proposed method fully utilizes the contextual information of images and introduces a simple but effective early-fusion strategy. The cooperation of these two components leads to the success of the effective camera-LiDAR fusion.
  arXiv  Detail & Related papers  (2021-08-17T08:53:11Z)
• SMD-Nets: Stereo Mixture Density Networks [68.56947049719936]
  We propose Stereo Mixture Density Networks (SMD-Nets), a simple yet effective learning framework compatible with a wide class of 2D and 3D architectures. Specifically, we exploit bimodal mixture densities as output representation and show that this allows for sharp and precise disparity estimates near discontinuities. We carry out comprehensive experiments on a new high-resolution and highly realistic synthetic stereo dataset, consisting of stereo pairs at 8Mpx resolution, as well as on real-world stereo datasets.
  arXiv  Detail & Related papers  (2021-04-08T16:15:46Z)
• SelfVoxeLO: Self-supervised LiDAR Odometry with Voxel-based Deep Neural Networks [81.64530401885476]
  We propose a self-supervised LiDAR odometry method, dubbed SelfVoxeLO, to tackle these two difficulties. Specifically, we propose a 3D convolution network to process the raw LiDAR data directly, which extracts features that better encode the 3D geometric patterns. We evaluate our method's performances on two large-scale datasets, i.e., KITTI and Apollo-SouthBay.
  arXiv  Detail & Related papers  (2020-10-19T09:23:39Z)
• Expanding Sparse Guidance for Stereo Matching [24.74333370941674]
  We propose a novel sparsity expansion technique to expand the sparse cues concerning RGB images for local feature enhancement. Our approach significantly boosts the existing state-of-the-art stereo algorithms with extremely sparse cues.
  arXiv  Detail & Related papers  (2020-04-24T06:41:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.