Efficient Spatial-Temporal Information Fusion for LiDAR-Based 3D Moving Object Segmentation

• URL: http://arxiv.org/abs/2207.02201v1
• Date: Tue, 5 Jul 2022 17:59:17 GMT
• Title: Efficient Spatial-Temporal Information Fusion for LiDAR-Based 3D Moving Object Segmentation
• Authors: Jiadai Sun, Yuchao Dai, Xianjing Zhang, Jintao Xu, Rui Ai, Weihao Gu, Xieyuanli Chen
• Abstract summary: We propose a novel deep neural network exploiting both spatial-temporal information and different representation modalities of LiDAR scans to improve LiDAR-MOS performance. Specifically, we first use a range image-based dual-branch structure to separately deal with spatial and temporal information. We also use a point refinement module via 3D sparse convolution to fuse the information from both LiDAR range image and point cloud representations.
• Score: 23.666607237164186
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurate moving object segmentation is an essential task for autonomous driving. It can provide effective information for many downstream tasks, such as collision avoidance, path planning, and static map construction. How to effectively exploit the spatial-temporal information is a critical question for 3D LiDAR moving object segmentation (LiDAR-MOS). In this work, we propose a novel deep neural network exploiting both spatial-temporal information and different representation modalities of LiDAR scans to improve LiDAR-MOS performance. Specifically, we first use a range image-based dual-branch structure to separately deal with spatial and temporal information that can be obtained from sequential LiDAR scans, and later combine them using motion-guided attention modules. We also use a point refinement module via 3D sparse convolution to fuse the information from both LiDAR range image and point cloud representations and reduce the artifacts on the borders of the objects. We verify the effectiveness of our proposed approach on the LiDAR-MOS benchmark of SemanticKITTI. Our method outperforms the state-of-the-art methods significantly in terms of LiDAR-MOS IoU. Benefiting from the devised coarse-to-fine architecture, our method operates online at sensor frame rate. The implementation of our method is available as open source at: https://github.com/haomo-ai/MotionSeg3D.

Related papers

• Future Does Matter: Boosting 3D Object Detection with Temporal Motion Estimation in Point Cloud Sequences [25.74000325019015]
  We introduce a novel LiDAR 3D object detection framework, namely LiSTM, to facilitate spatial-temporal feature learning with cross-frame motion forecasting information. We have conducted experiments on the aggregation and nuScenes datasets to demonstrate that the proposed framework achieves superior 3D detection performance.
  arXiv  Detail & Related papers  (2024-09-06T16:29:04Z)
• FAST-LIVO2: Fast, Direct LiDAR-Inertial-Visual Odometry [28.606325312582218]
  We propose FAST-LIVO2, a fast, direct LiDAR-inertial-visual odometry framework to achieve accurate and robust state estimation in SLAM tasks. FAST-LIVO2 fuses the IMU, LiDAR and image measurements efficiently through a sequential update strategy. We show three applications of FAST-LIVO2, including real-time onboard navigation, airborne mapping, and 3D model rendering.
  arXiv  Detail & Related papers  (2024-08-26T06:01:54Z)
• TASeg: Temporal Aggregation Network for LiDAR Semantic Segmentation [80.13343299606146]
  We propose a Temporal LiDAR Aggregation and Distillation (TLAD) algorithm, which leverages historical priors to assign different aggregation steps for different classes. To make full use of temporal images, we design a Temporal Image Aggregation and Fusion (TIAF) module, which can greatly expand the camera FOV. We also develop a Static-Moving Switch Augmentation (SMSA) algorithm, which utilizes sufficient temporal information to enable objects to switch their motion states freely.
  arXiv  Detail & Related papers  (2024-07-13T03:00:16Z)
• Multi-Modal Data-Efficient 3D Scene Understanding for Autonomous Driving [58.16024314532443]
  We introduce LaserMix++, a framework that integrates laser beam manipulations from disparate LiDAR scans and incorporates LiDAR-camera correspondences to assist data-efficient learning. Results demonstrate that LaserMix++ outperforms fully supervised alternatives, achieving comparable accuracy with five times fewer annotations. This substantial advancement underscores the potential of semi-supervised approaches in reducing the reliance on extensive labeled data in LiDAR-based 3D scene understanding systems.
  arXiv  Detail & Related papers  (2024-05-08T17:59:53Z)
• LiDAR-BEVMTN: Real-Time LiDAR Bird's-Eye View Multi-Task Perception Network for Autonomous Driving [12.713417063678335]
  We present a real-time multi-task convolutional neural network for LiDAR-based object detection, semantics, and motion segmentation. We propose a novel Semantic Weighting and Guidance (SWAG) module to transfer semantic features for improved object detection selectively. We achieve state-of-the-art results for two tasks, semantic and motion segmentation, and close to state-of-the-art performance for 3D object detection.
  arXiv  Detail & Related papers  (2023-07-17T21:22:17Z)
• SUIT: Learning Significance-guided Information for 3D Temporal Detection [15.237488449422008]
  We learn Significance-gUided Information for 3D Temporal detection (SUIT), which simplifies temporal information as sparse features for information fusion across frames. We evaluate our method on large-scale nuScenes and dataset, where our SUIT not only significantly reduces the memory and cost of temporal fusion, but also performs well over the state-of-the-art baselines.
  arXiv  Detail & Related papers  (2023-07-04T16:22:10Z)
• MotionBEV: Attention-Aware Online LiDAR Moving Object Segmentation with Bird's Eye View based Appearance and Motion Features [5.186531650935954]
  We present MotionBEV, a fast and accurate framework for LiDAR moving object segmentation. Our approach converts 3D LiDAR scans into a 2D polar BEV representation to improve computational efficiency. We employ a dual-branch network bridged by the Appearance-Motion Co-attention Module (AMCM) to adaptively fuse the LiDAR-temporal information from appearance and motion features.
  arXiv  Detail & Related papers  (2023-05-12T09:28:09Z)
• Ret3D: Rethinking Object Relations for Efficient 3D Object Detection in Driving Scenes [82.4186966781934]
  We introduce a simple, efficient, and effective two-stage detector, termed as Ret3D. At the core of Ret3D is the utilization of novel intra-frame and inter-frame relation modules. With negligible extra overhead, Ret3D achieves the state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-08-18T03:48:58Z)
• Benchmarking the Robustness of LiDAR-Camera Fusion for 3D Object Detection [58.81316192862618]
  Two critical sensors for 3D perception in autonomous driving are the camera and the LiDAR. fusing these two modalities can significantly boost the performance of 3D perception models. We benchmark the state-of-the-art fusion methods for the first time.
  arXiv  Detail & Related papers  (2022-05-30T09:35:37Z)
• LiDAR-based 4D Panoptic Segmentation via Dynamic Shifting Network [56.71765153629892]
  We propose the Dynamic Shifting Network (DS-Net), which serves as an effective panoptic segmentation framework in the point cloud realm. Our proposed DS-Net achieves superior accuracies over current state-of-the-art methods in both tasks. We extend DS-Net to 4D panoptic LiDAR segmentation by the temporally unified instance clustering on aligned LiDAR frames.
  arXiv  Detail & Related papers  (2022-03-14T15:25:42Z)
• Learning Moving-Object Tracking with FMCW LiDAR [53.05551269151209]
  We propose a learning-based moving-object tracking method utilizing our newly developed LiDAR sensor, Frequency Modulated Continuous Wave (FMCW) LiDAR. Given the labels, we propose a contrastive learning framework, which pulls together the features from the same instance in embedding space and pushes apart the features from different instances to improve the tracking quality.
  arXiv  Detail & Related papers  (2022-03-02T09:11:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.