Point Cloud Mamba: Point Cloud Learning via State Space Model

• URL: http://arxiv.org/abs/2403.00762v4
• Date: Fri, 11 Oct 2024 07:27:00 GMT
• Title: Point Cloud Mamba: Point Cloud Learning via State Space Model
• Authors: Tao Zhang, Haobo Yuan, Lu Qi, Jiangning Zhang, Qianyu Zhou, Shunping Ji, Shuicheng Yan, Xiangtai Li,
• Abstract summary: We show that Mamba-based point cloud methods can outperform previous methods based on transformer or multi-layer perceptrons (MLPs) In particular, we demonstrate that Mamba-based point cloud methods can outperform previous methods based on transformer or multi-layer perceptrons (MLPs) Point Cloud Mamba surpasses the state-of-the-art (SOTA) point-based method PointNeXt and achieves new SOTA performance on the ScanNN, ModelNet40, ShapeNetPart, and S3DIS datasets.
• Score: 73.7454734756626
• License:
• Abstract: Recently, state space models have exhibited strong global modeling capabilities and linear computational complexity in contrast to transformers. This research focuses on applying such architecture to more efficiently and effectively model point cloud data globally with linear computational complexity. In particular, for the first time, we demonstrate that Mamba-based point cloud methods can outperform previous methods based on transformer or multi-layer perceptrons (MLPs). To enable Mamba to process 3-D point cloud data more effectively, we propose a novel Consistent Traverse Serialization method to convert point clouds into 1-D point sequences while ensuring that neighboring points in the sequence are also spatially adjacent. Consistent Traverse Serialization yields six variants by permuting the order of \textit{x}, \textit{y}, and \textit{z} coordinates, and the synergistic use of these variants aids Mamba in comprehensively observing point cloud data. Furthermore, to assist Mamba in handling point sequences with different orders more effectively, we introduce point prompts to inform Mamba of the sequence's arrangement rules. Finally, we propose positional encoding based on spatial coordinate mapping to inject positional information into point cloud sequences more effectively. Point Cloud Mamba surpasses the state-of-the-art (SOTA) point-based method PointNeXt and achieves new SOTA performance on the ScanObjectNN, ModelNet40, ShapeNetPart, and S3DIS datasets. It is worth mentioning that when using a more powerful local feature extraction module, our PCM achieves 79.6 mIoU on S3DIS, significantly surpassing the previous SOTA models, DeLA and PTv3, by 5.5 mIoU and 4.9 mIoU, respectively.

Related papers

• NIMBA: Towards Robust and Principled Processing of Point Clouds With SSMs [9.978766637766373]
  We introduce a method to convert point clouds into 1D sequences that maintain 3D spatial structure with no need for data replication. Our method does not require positional embeddings and allows for shorter sequence lengths while still achieving state-of-the-art results.
  arXiv  Detail & Related papers  (2024-10-31T18:58:40Z)
• Serialized Point Mamba: A Serialized Point Cloud Mamba Segmentation Model [9.718016281821471]
  Serialized Point Cloud Mamba Model (Serialized Point Mamba) developed. Inspired by the Mamba model's success in natural language processing, we propose the Serialized Point Cloud Mamba Model. Method achieved 76.8 mIoU on Scannet and facilitating 70.3 mIoU on S3DIS.
  arXiv  Detail & Related papers  (2024-07-17T05:26:58Z)
• Mamba24/8D: Enhancing Global Interaction in Point Clouds via State Space Model [37.375866491592305]
  We introduce Mamba, a SSM-based architecture, to the point cloud domain. We propose Mamba24/8D, which has strong global modeling capability under linear complexity. Mamba24/8D obtains state of the art results on several 3D point cloud segmentation tasks.
  arXiv  Detail & Related papers  (2024-06-25T10:23:53Z)
• Mamba3D: Enhancing Local Features for 3D Point Cloud Analysis via State Space Model [18.30032389736101]
  Mamba model, based on state space models (SSM), outperforms Transformer in multiple areas with only linear complexity. We present Mamba3D, a state space model tailored for point cloud learning to enhance local feature extraction.
  arXiv  Detail & Related papers  (2024-04-23T12:20:27Z)
• 3DMambaComplete: Exploring Structured State Space Model for Point Cloud Completion [19.60626235337542]
  3DMambaComplete is a point cloud completion network built on the novel Mamba framework. It encodes point cloud features using Mamba's selection mechanism and predicts a set of Hyperpoints. A deformation method transforms the 2D mesh representation of HyperPoints into a fine-grained 3D structure for point cloud reconstruction.
  arXiv  Detail & Related papers  (2024-04-10T15:45:03Z)
• PointMamba: A Simple State Space Model for Point Cloud Analysis [65.59944745840866]
  We propose PointMamba, transferring the success of Mamba, a recent representative state space model (SSM), from NLP to point cloud analysis tasks. Unlike traditional Transformers, PointMamba employs a linear complexity algorithm, presenting global modeling capacity while significantly reducing computational costs.
  arXiv  Detail & Related papers  (2024-02-16T14:56:13Z)
• PointPatchMix: Point Cloud Mixing with Patch Scoring [58.58535918705736]
  We propose PointPatchMix, which mixes point clouds at the patch level and generates content-based targets for mixed point clouds. Our approach preserves local features at the patch level, while the patch scoring module assigns targets based on the content-based significance score from a pre-trained teacher model. With Point-MAE as our baseline, our model surpasses previous methods by a significant margin, achieving 86.3% accuracy on ScanObjectNN and 94.1% accuracy on ModelNet40.
  arXiv  Detail & Related papers  (2023-03-12T14:49:42Z)
• CloudAttention: Efficient Multi-Scale Attention Scheme For 3D Point Cloud Learning [81.85951026033787]
  We set transformers in this work and incorporate them into a hierarchical framework for shape classification and part and scene segmentation. We also compute efficient and dynamic global cross attentions by leveraging sampling and grouping at each iteration. The proposed hierarchical model achieves state-of-the-art shape classification in mean accuracy and yields results on par with the previous segmentation methods.
  arXiv  Detail & Related papers  (2022-07-31T21:39:15Z)
• Dual Adaptive Transformations for Weakly Supervised Point Cloud Segmentation [78.6612285236938]
  We propose a novel DAT (textbfDual textbfAdaptive textbfTransformations) model for weakly supervised point cloud segmentation. We evaluate our proposed DAT model with two popular backbones on the large-scale S3DIS and ScanNet-V2 datasets.
  arXiv  Detail & Related papers  (2022-07-19T05:43:14Z)
• Learning Semantic Segmentation of Large-Scale Point Clouds with Random Sampling [52.464516118826765]
  We introduce RandLA-Net, an efficient and lightweight neural architecture to infer per-point semantics for large-scale point clouds. The key to our approach is to use random point sampling instead of more complex point selection approaches. Our RandLA-Net can process 1 million points in a single pass up to 200x faster than existing approaches.
  arXiv  Detail & Related papers  (2021-07-06T05:08:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.