Point Mamba: A Novel Point Cloud Backbone Based on State Space Model with Octree-Based Ordering Strategy

• URL: http://arxiv.org/abs/2403.06467v2
• Date: Mon, 18 Mar 2024 02:56:26 GMT
• Title: Point Mamba: A Novel Point Cloud Backbone Based on State Space Model with Octree-Based Ordering Strategy
• Authors: Jiuming Liu, Ruiji Yu, Yian Wang, Yu Zheng, Tianchen Deng, Weicai Ye, Hesheng Wang,
• Abstract summary: We propose a novel SSM-based point cloud processing backbone, named Point Mamba, with a causality-aware ordering mechanism. Our method achieves state-of-the-art performance compared with transformer-based counterparts, with 93.4% accuracy and 75.7 mIOU respectively. Our method demonstrates the great potential that SSM can serve as a generic backbone in point cloud understanding.
• Score: 15.032048930130614
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, state space model (SSM) has gained great attention due to its promising performance, linear complexity, and long sequence modeling ability in both language and image domains. However, it is non-trivial to extend SSM to the point cloud field, because of the causality requirement of SSM and the disorder and irregularity nature of point clouds. In this paper, we propose a novel SSM-based point cloud processing backbone, named Point Mamba, with a causality-aware ordering mechanism. To construct the causal dependency relationship, we design an octree-based ordering strategy on raw irregular points, globally sorting points in a z-order sequence and also retaining their spatial proximity. Our method achieves state-of-the-art performance compared with transformer-based counterparts, with 93.4% accuracy and 75.7 mIOU respectively on the ModelNet40 classification dataset and ScanNet semantic segmentation dataset. Furthermore, our Point Mamba has linear complexity, which is more efficient than transformer-based methods. Our method demonstrates the great potential that SSM can serve as a generic backbone in point cloud understanding. Codes are released at https://github.com/IRMVLab/Point-Mamba.

Related papers

• Serialized Point Mamba: A Serialized Point Cloud Mamba Segmentation Model [9.718016281821471]
  Serialized Point Cloud Mamba Model (Serialized Point Mamba) developed. Inspired by the Mamba model's success in natural language processing, we propose the Serialized Point Cloud Mamba Model. Method achieved 76.8 mIoU on Scannet and facilitating 70.3 mIoU on S3DIS.
  arXiv  Detail & Related papers  (2024-07-17T05:26:58Z)
• Mamba24/8D: Enhancing Global Interaction in Point Clouds via State Space Model [37.375866491592305]
  We introduce Mamba, a SSM-based architecture, to the point cloud domain. We propose Mamba24/8D, which has strong global modeling capability under linear complexity. Mamba24/8D obtains state of the art results on several 3D point cloud segmentation tasks.
  arXiv  Detail & Related papers  (2024-06-25T10:23:53Z)
• Mamba3D: Enhancing Local Features for 3D Point Cloud Analysis via State Space Model [18.30032389736101]
  Mamba model, based on state space models (SSM), outperforms Transformer in multiple areas with only linear complexity. We present Mamba3D, a state space model tailored for point cloud learning to enhance local feature extraction.
  arXiv  Detail & Related papers  (2024-04-23T12:20:27Z)
• Point Cloud Mamba: Point Cloud Learning via State Space Model [64.85865751243448]
  This research focuses on applying such architecture in point cloud analysis. We demonstrate that Mamba-based point cloud methods can outperform previous methods based on transformer or multi-layer perceptrons (MLPs) Point Cloud Mamba surpasses the state-of-the-art (SOTA) point-based method PointNeXt and achieves new SOTA performance on the ScanObjectNN, ModelNet40, ShapeNetPart, and S3DIS datasets.
  arXiv  Detail & Related papers  (2024-03-01T18:59:03Z)
• PointMamba: A Simple State Space Model for Point Cloud Analysis [65.59944745840866]
  We propose PointMamba, transferring the success of Mamba, a recent representative state space model (SSM), from NLP to point cloud analysis tasks. Unlike traditional Transformers, PointMamba employs a linear complexity algorithm, presenting global modeling capacity while significantly reducing computational costs.
  arXiv  Detail & Related papers  (2024-02-16T14:56:13Z)
• PointPatchMix: Point Cloud Mixing with Patch Scoring [58.58535918705736]
  We propose PointPatchMix, which mixes point clouds at the patch level and generates content-based targets for mixed point clouds. Our approach preserves local features at the patch level, while the patch scoring module assigns targets based on the content-based significance score from a pre-trained teacher model. With Point-MAE as our baseline, our model surpasses previous methods by a significant margin, achieving 86.3% accuracy on ScanObjectNN and 94.1% accuracy on ModelNet40.
  arXiv  Detail & Related papers  (2023-03-12T14:49:42Z)
• CloudAttention: Efficient Multi-Scale Attention Scheme For 3D Point Cloud Learning [81.85951026033787]
  We set transformers in this work and incorporate them into a hierarchical framework for shape classification and part and scene segmentation. We also compute efficient and dynamic global cross attentions by leveraging sampling and grouping at each iteration. The proposed hierarchical model achieves state-of-the-art shape classification in mean accuracy and yields results on par with the previous segmentation methods.
  arXiv  Detail & Related papers  (2022-07-31T21:39:15Z)
• Dynamic Convolution for 3D Point Cloud Instance Segmentation [146.7971476424351]
  We propose an approach to instance segmentation from 3D point clouds based on dynamic convolution. We gather homogeneous points that have identical semantic categories and close votes for the geometric centroids. The proposed approach is proposal-free, and instead exploits a convolution process that adapts to the spatial and semantic characteristics of each instance.
  arXiv  Detail & Related papers  (2021-07-18T09:05:16Z)
• ASAP-Net: Attention and Structure Aware Point Cloud Sequence Segmentation [49.15948235059343]
  We further improve point-temporal cloud feature with a flexible module called ASAP. Our ASAP module contains an attentive temporal embedding layer to fuse the relatively informative local features across frames in a recurrent fashion. We show the generalization ability of the proposed ASAP module with different computation backbone networks for point cloud sequence segmentation.
  arXiv  Detail & Related papers  (2020-08-12T07:37:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.