Multi-Space Alignments Towards Universal LiDAR Segmentation

• URL: http://arxiv.org/abs/2405.01538v1
• Date: Thu, 2 May 2024 17:59:57 GMT
• Title: Multi-Space Alignments Towards Universal LiDAR Segmentation
• Authors: Youquan Liu, Lingdong Kong, Xiaoyang Wu, Runnan Chen, Xin Li, Liang Pan, Ziwei Liu, Yuexin Ma,
• Abstract summary: M3Net is a one-of-a-kind framework for fulfilling multi-task, multi-dataset, multi-modality LiDAR segmentation. We first combine large-scale driving datasets acquired by different types of sensors from diverse scenes. We then conduct alignments in three spaces, namely data, feature, and label spaces, during the training.
• Score: 50.992103482269016
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: A unified and versatile LiDAR segmentation model with strong robustness and generalizability is desirable for safe autonomous driving perception. This work presents M3Net, a one-of-a-kind framework for fulfilling multi-task, multi-dataset, multi-modality LiDAR segmentation in a universal manner using just a single set of parameters. To better exploit data volume and diversity, we first combine large-scale driving datasets acquired by different types of sensors from diverse scenes and then conduct alignments in three spaces, namely data, feature, and label spaces, during the training. As a result, M3Net is capable of taming heterogeneous data for training state-of-the-art LiDAR segmentation models. Extensive experiments on twelve LiDAR segmentation datasets verify our effectiveness. Notably, using a shared set of parameters, M3Net achieves 75.1%, 83.1%, and 72.4% mIoU scores, respectively, on the official benchmarks of SemanticKITTI, nuScenes, and Waymo Open.

Related papers

• 3DLabelProp: Geometric-Driven Domain Generalization for LiDAR Semantic Segmentation in Autonomous Driving [7.35996217853436]
  Domain generalization aims to find ways for deep learning models to maintain their performance despite domain shifts between training and inference datasets. This is particularly important for models that need to be robust or are costly to train. This work proposes a geometry-based approach, leveraging the sequential structure of LiDAR sensors, which sets it apart from the learning-based methods commonly found in the literature.
  arXiv  Detail & Related papers  (2025-01-24T16:22:35Z)
• LiMoE: Mixture of LiDAR Representation Learners from Automotive Scenes [55.33167217384738]
  LiMoE is a framework that integrates the Mixture of Experts (MoE) paradigm into LiDAR data representation learning. Our approach consists of three stages: Image-to-LiDAR Pretraining, Contrastive Mixture Learning (CML), and Semantic Mixture Supervision (SMS)
  arXiv  Detail & Related papers  (2025-01-07T18:59:58Z)
• SM3Det: A Unified Model for Multi-Modal Remote Sensing Object Detection [73.49799596304418]
  This paper introduces a new task called Multi-Modal datasets and Multi-Task Object Detection (M2Det) for remote sensing. It is designed to accurately detect horizontal or oriented objects from any sensor modality. This task poses challenges due to 1) the trade-offs involved in managing multi-modal modelling and 2) the complexities of multi-task optimization.
  arXiv  Detail & Related papers  (2024-12-30T02:47:51Z)
• Uni$^2$Det: Unified and Universal Framework for Prompt-Guided Multi-dataset 3D Detection [64.08296187555095]
  Uni$2$Det is a framework for unified and universal multi-dataset training on 3D detection. We introduce multi-stage prompting modules for multi-dataset 3D detection. Results on zero-shot cross-dataset transfer validate the generalization capability of our proposed method.
  arXiv  Detail & Related papers  (2024-09-30T17:57:50Z)
• MergeOcc: Bridge the Domain Gap between Different LiDARs for Robust Occupancy Prediction [8.993992124170624]
  MergeOcc is developed to simultaneously handle different LiDARs by leveraging multiple datasets. The effectiveness of MergeOcc is validated through experiments on two prominent datasets for autonomous vehicles.
  arXiv  Detail & Related papers  (2024-03-13T13:23:05Z)
• Single-Model and Any-Modality for Video Object Tracking [85.83753760853142]
  We introduce Un-Track, a Unified Tracker of a single set of parameters for any modality. To handle any modality, our method learns their common latent space through low-rank factorization and reconstruction techniques. Our Un-Track achieves +8.1 absolute F-score gain, on the DepthTrack dataset, by introducing only +2.14 (over 21.50) GFLOPs with +6.6M (over 93M) parameters.
  arXiv  Detail & Related papers  (2023-11-27T14:17:41Z)
• MV-JAR: Masked Voxel Jigsaw and Reconstruction for LiDAR-Based Self-Supervised Pre-Training [58.07391711548269]
  Masked Voxel Jigsaw and Reconstruction (MV-JAR) method for LiDAR-based self-supervised pre-training. Masked Voxel Jigsaw and Reconstruction (MV-JAR) method for LiDAR-based self-supervised pre-training.
  arXiv  Detail & Related papers  (2023-03-23T17:59:02Z)
• Shared Manifold Learning Using a Triplet Network for Multiple Sensor Translation and Fusion with Missing Data [2.452410403088629]
  We propose a Contrastive learning based MultiModal Alignment Network (CoMMANet) to align data from different sensors into a shared and discriminative manifold. The proposed architecture uses a multimodal triplet autoencoder to cluster the latent space in such a way that samples of the same classes from each heterogeneous modality are mapped close to each other.
  arXiv  Detail & Related papers  (2022-10-25T20:22:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.