3D Learnable Supertoken Transformer for LiDAR Point Cloud Scene Segmentation

• URL: http://arxiv.org/abs/2405.15826v1
• Date: Thu, 23 May 2024 20:41:15 GMT
• Title: 3D Learnable Supertoken Transformer for LiDAR Point Cloud Scene Segmentation
• Authors: Dening Lu, Jun Zhou, Kyle Gao, Linlin Xu, Jonathan Li,
• Abstract summary: This paper proposes a novel 3D Transformer framework, named 3D Learnable Supertoken Transformer (3DLST) The 3DLST is equipped with a novel W-net architecture instead of the common U-net design. It achieves satisfactory results in terms of algorithm efficiency, which is up to 5x faster than previous best-performing methods.
• Score: 19.94836580257577
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: 3D Transformers have achieved great success in point cloud understanding and representation. However, there is still considerable scope for further development in effective and efficient Transformers for large-scale LiDAR point cloud scene segmentation. This paper proposes a novel 3D Transformer framework, named 3D Learnable Supertoken Transformer (3DLST). The key contributions are summarized as follows. Firstly, we introduce the first Dynamic Supertoken Optimization (DSO) block for efficient token clustering and aggregating, where the learnable supertoken definition avoids the time-consuming pre-processing of traditional superpoint generation. Since the learnable supertokens can be dynamically optimized by multi-level deep features during network learning, they are tailored to the semantic homogeneity-aware token clustering. Secondly, an efficient Cross-Attention-guided Upsampling (CAU) block is proposed for token reconstruction from optimized supertokens. Thirdly, the 3DLST is equipped with a novel W-net architecture instead of the common U-net design, which is more suitable for Transformer-based feature learning. The SOTA performance on three challenging LiDAR datasets (airborne MultiSpectral LiDAR (MS-LiDAR) (89.3% of the average F1 score), DALES (80.2% of mIoU), and Toronto-3D dataset (80.4% of mIoU)) demonstrate the superiority of 3DLST and its strong adaptability to various LiDAR point cloud data (airborne MS-LiDAR, aerial LiDAR, and vehicle-mounted LiDAR data). Furthermore, 3DLST also achieves satisfactory results in terms of algorithm efficiency, which is up to 5x faster than previous best-performing methods.

Related papers

• Robin3D: Improving 3D Large Language Model via Robust Instruction Tuning [55.339257446600634]
  We introduce Robin3D, a powerful 3DLLM trained on large-scale instruction-following data. We construct 1 million instruction-following data, consisting of 344K Adversarial samples, 508K Diverse samples, and 165K benchmark training set samples. Robin3D consistently outperforms previous methods across five widely-used 3D multimodal learning benchmarks.
  arXiv  Detail & Related papers  (2024-09-30T21:55:38Z)
• Unleashing the Potential of Mamba: Boosting a LiDAR 3D Sparse Detector by Using Cross-Model Knowledge Distillation [22.653014803666668]
  We propose a Faster LiDAR 3D object detection framework, called FASD, which implements heterogeneous model distillation by adaptively uniform cross-model voxel features. We aim to distill the transformer's capacity for high-performance sequence modeling into Mamba models with low FLOPs, achieving a significant improvement in accuracy through knowledge transfer. We evaluated the framework on datasets and nuScenes, achieving a 4x reduction in resource consumption and a 1-2% performance improvement over the current SoTA methods.
  arXiv  Detail & Related papers  (2024-09-17T09:30:43Z)
• Efficient Point Transformer with Dynamic Token Aggregating for Point Cloud Processing [19.73918716354272]
  We propose an efficient point TransFormer with Dynamic Token Aggregating (DTA-Former) for point cloud representation and processing. It achieves SOTA performance with up to 30$times$ faster than prior point Transformers on ModelNet40, ShapeNet, and airborne MultiSpectral LiDAR (MS-LiDAR) datasets.
  arXiv  Detail & Related papers  (2024-05-23T20:50:50Z)
• Multi-Modal Data-Efficient 3D Scene Understanding for Autonomous Driving [58.16024314532443]
  We introduce LaserMix++, a framework that integrates laser beam manipulations from disparate LiDAR scans and incorporates LiDAR-camera correspondences to assist data-efficient learning. Results demonstrate that LaserMix++ outperforms fully supervised alternatives, achieving comparable accuracy with five times fewer annotations. This substantial advancement underscores the potential of semi-supervised approaches in reducing the reliance on extensive labeled data in LiDAR-based 3D scene understanding systems.
  arXiv  Detail & Related papers  (2024-05-08T17:59:53Z)
• Point Cloud Mamba: Point Cloud Learning via State Space Model [73.7454734756626]
  We show that Mamba-based point cloud methods can outperform previous methods based on transformer or multi-layer perceptrons (MLPs) In particular, we demonstrate that Mamba-based point cloud methods can outperform previous methods based on transformer or multi-layer perceptrons (MLPs) Point Cloud Mamba surpasses the state-of-the-art (SOTA) point-based method PointNeXt and achieves new SOTA performance on the ScanNN, ModelNet40, ShapeNetPart, and S3DIS datasets.
  arXiv  Detail & Related papers  (2024-03-01T18:59:03Z)
• Hourglass Tokenizer for Efficient Transformer-Based 3D Human Pose Estimation [73.31524865643709]
  We present a plug-and-play pruning-and-recovering framework, called Hourglass Tokenizer (HoT), for efficient transformer-based 3D pose estimation from videos. Our HoDT begins with pruning pose tokens of redundant frames and ends with recovering full-length tokens, resulting in a few pose tokens in the intermediate transformer blocks. Our method can achieve both high efficiency and estimation accuracy compared to the original VPT models.
  arXiv  Detail & Related papers  (2023-11-20T18:59:51Z)
• LiDARFormer: A Unified Transformer-based Multi-task Network for LiDAR Perception [15.919789515451615]
  We introduce a new LiDAR multi-task learning paradigm based on the transformer. LiDARFormer exploits cross-task synergy to boost the performance of LiDAR perception tasks. LiDARFormer is evaluated on the large-scale nuScenes and the Open datasets for both 3D detection and semantic segmentation tasks.
  arXiv  Detail & Related papers  (2023-03-21T20:52:02Z)
• Li3DeTr: A LiDAR based 3D Detection Transformer [0.0]
  Li3DeTr is an end-to-end LiDAR based 3D Detection Transformer for autonomous driving. The Li3DeTr network achieves 61.3% mAP and 67.6% NDS.
  arXiv  Detail & Related papers  (2022-10-27T12:23:54Z)
• CloudAttention: Efficient Multi-Scale Attention Scheme For 3D Point Cloud Learning [81.85951026033787]
  We set transformers in this work and incorporate them into a hierarchical framework for shape classification and part and scene segmentation. We also compute efficient and dynamic global cross attentions by leveraging sampling and grouping at each iteration. The proposed hierarchical model achieves state-of-the-art shape classification in mean accuracy and yields results on par with the previous segmentation methods.
  arXiv  Detail & Related papers  (2022-07-31T21:39:15Z)
• LiDAR-based 4D Panoptic Segmentation via Dynamic Shifting Network [56.71765153629892]
  We propose the Dynamic Shifting Network (DS-Net), which serves as an effective panoptic segmentation framework in the point cloud realm. Our proposed DS-Net achieves superior accuracies over current state-of-the-art methods in both tasks. We extend DS-Net to 4D panoptic LiDAR segmentation by the temporally unified instance clustering on aligned LiDAR frames.
  arXiv  Detail & Related papers  (2022-03-14T15:25:42Z)
• 3DCTN: 3D Convolution-Transformer Network for Point Cloud Classification [23.0009969537045]
  This paper presents a novel hierarchical framework that incorporates convolution with Transformer for point cloud classification. Our method achieves state-of-the-art classification performance, in terms of both accuracy and efficiency.
  arXiv  Detail & Related papers  (2022-03-02T02:42:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.