Vision-based Large-scale 3D Semantic Mapping for Autonomous Driving Applications

• URL: http://arxiv.org/abs/2203.01087v1
• Date: Wed, 2 Mar 2022 13:18:38 GMT
• Title: Vision-based Large-scale 3D Semantic Mapping for Autonomous Driving Applications
• Authors: Qing Cheng, Niclas Zeller, Daniel Cremers
• Abstract summary: We present a complete pipeline for 3D semantic mapping solely based on a stereo camera system. The pipeline comprises a direct visual odometry front-end as well as a back-end for global temporal integration. We propose a simple but effective voting scheme which improves the quality and consistency of the 3D point labels.
• Score: 53.553924052102126
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we present a complete pipeline for 3D semantic mapping solely based on a stereo camera system. The pipeline comprises a direct sparse visual odometry front-end as well as a back-end for global optimization including GNSS integration, and semantic 3D point cloud labeling. We propose a simple but effective temporal voting scheme which improves the quality and consistency of the 3D point labels. Qualitative and quantitative evaluations of our pipeline are performed on the KITTI-360 dataset. The results show the effectiveness of our proposed voting scheme and the capability of our pipeline for efficient large-scale 3D semantic mapping. The large-scale mapping capabilities of our pipeline is furthermore demonstrated by presenting a very large-scale semantic map covering 8000 km of roads generated from data collected by a fleet of vehicles.

Related papers

• LISNeRF Mapping: LiDAR-based Implicit Mapping via Semantic Neural Fields for Large-Scale 3D Scenes [2.822816116516042]
  Large-scale semantic mapping is crucial for outdoor autonomous agents to fulfill high-level tasks such as planning and navigation. This paper proposes a novel method for large-scale 3D semantic reconstruction through implicit representations from posed LiDAR measurements alone.
  arXiv  Detail & Related papers  (2023-11-04T03:55:38Z)
• Volumetric Semantically Consistent 3D Panoptic Mapping [77.13446499924977]
  We introduce an online 2D-to-3D semantic instance mapping algorithm aimed at generating semantic 3D maps suitable for autonomous agents in unstructured environments. It introduces novel ways of integrating semantic prediction confidence during mapping, producing semantic and instance-consistent 3D regions. The proposed method achieves accuracy superior to the state of the art on public large-scale datasets, improving on a number of widely used metrics.
  arXiv  Detail & Related papers  (2023-09-26T08:03:10Z)
• PointOcc: Cylindrical Tri-Perspective View for Point-based 3D Semantic Occupancy Prediction [72.75478398447396]
  We propose a cylindrical tri-perspective view to represent point clouds effectively and comprehensively. Considering the distance distribution of LiDAR point clouds, we construct the tri-perspective view in the cylindrical coordinate system. We employ spatial group pooling to maintain structural details during projection and adopt 2D backbones to efficiently process each TPV plane.
  arXiv  Detail & Related papers  (2023-08-31T17:57:17Z)
• SimpleMapping: Real-Time Visual-Inertial Dense Mapping with Deep Multi-View Stereo [13.535871843518953]
  We present a real-time visual-inertial dense mapping method with high quality using only monocular images and IMU readings. We propose a sparse point aided stereo neural network (SPA-MVSNet) that can effectively leverage the informative but noisy sparse points from the VIO system. Our proposed dense mapping system achieves a 39.7% improvement in F-score over existing systems when evaluated on the challenging scenarios of the EuRoC dataset.
  arXiv  Detail & Related papers  (2023-06-14T17:28:45Z)
• TANDEM: Tracking and Dense Mapping in Real-time using Deep Multi-view Stereo [55.30992853477754]
  We present TANDEM, a real-time monocular tracking and dense framework. For pose estimation, TANDEM performs photometric bundle adjustment based on a sliding window of alignments. TANDEM shows state-of-the-art real-time 3D reconstruction performance.
  arXiv  Detail & Related papers  (2021-11-14T19:01:02Z)
• Learnable Online Graph Representations for 3D Multi-Object Tracking [156.58876381318402]
  We propose a unified and learning based approach to the 3D MOT problem. We employ a Neural Message Passing network for data association that is fully trainable. We show the merit of the proposed approach on the publicly available nuScenes dataset by achieving state-of-the-art performance of 65.6% AMOTA and 58% fewer ID-switches.
  arXiv  Detail & Related papers  (2021-04-23T17:59:28Z)
• Crowdsourced 3D Mapping: A Combined Multi-View Geometry and Self-Supervised Learning Approach [10.610403488989428]
  We propose a framework that estimates the 3D positions of semantically meaningful landmarks without assuming known camera intrinsics. We utilize multi-view geometry as well as deep learning based self-calibration, depth, and ego-motion estimation for traffic sign positioning. We achieve an average single-journey relative and absolute positioning accuracy of 39cm and 1.26m respectively.
  arXiv  Detail & Related papers  (2020-07-25T12:10:16Z)
• Predicting Semantic Map Representations from Images using Pyramid Occupancy Networks [27.86228863466213]
  We present a simple, unified approach for estimating maps directly from monocular images using a single end-to-end deep learning architecture. We demonstrate the effectiveness of our approach by evaluating against several challenging baselines on the NuScenes and Argoverse datasets.
  arXiv  Detail & Related papers  (2020-03-30T12:39:44Z)
• OmniSLAM: Omnidirectional Localization and Dense Mapping for Wide-baseline Multi-camera Systems [88.41004332322788]
  We present an omnidirectional localization and dense mapping system for a wide-baseline multiview stereo setup with ultra-wide field-of-view (FOV) fisheye cameras. For more practical and accurate reconstruction, we first introduce improved and light-weighted deep neural networks for the omnidirectional depth estimation. We integrate our omnidirectional depth estimates into the visual odometry (VO) and add a loop closing module for global consistency.
  arXiv  Detail & Related papers  (2020-03-18T05:52:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.