Multi-scale Geometry-aware Transformer for 3D Point Cloud Classification

• URL: http://arxiv.org/abs/2304.05694v1
• Date: Wed, 12 Apr 2023 08:34:56 GMT
• Title: Multi-scale Geometry-aware Transformer for 3D Point Cloud Classification
• Authors: Xian Wei, Muyu Wang, Shing-Ho Jonathan Lin, Zhengyu Li, Jian Yang, Arafat Al-Jawari, Xuan Tang
• Abstract summary: This paper proposes a self-attention plug-in module with its variants, Multi-scale Geometry-aware Transformer (MGT) MGT processes point cloud data with multi-scale local and global geometric information in the following three aspects. Experimental results demonstrate that the MGT vastly increases the capability of capturing multi-scale geometry using the self-attention mechanism.
• Score: 17.836838702265332
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Self-attention modules have demonstrated remarkable capabilities in capturing long-range relationships and improving the performance of point cloud tasks. However, point cloud objects are typically characterized by complex, disordered, and non-Euclidean spatial structures with multiple scales, and their behavior is often dynamic and unpredictable. The current self-attention modules mostly rely on dot product multiplication and dimension alignment among query-key-value features, which cannot adequately capture the multi-scale non-Euclidean structures of point cloud objects. To address these problems, this paper proposes a self-attention plug-in module with its variants, Multi-scale Geometry-aware Transformer (MGT). MGT processes point cloud data with multi-scale local and global geometric information in the following three aspects. At first, the MGT divides point cloud data into patches with multiple scales. Secondly, a local feature extractor based on sphere mapping is proposed to explore the geometry inner each patch and generate a fixed-length representation for each patch. Thirdly, the fixed-length representations are fed into a novel geodesic-based self-attention to capture the global non-Euclidean geometry between patches. Finally, all the modules are integrated into the framework of MGT with an end-to-end training scheme. Experimental results demonstrate that the MGT vastly increases the capability of capturing multi-scale geometry using the self-attention mechanism and achieves strong competitive performance on mainstream point cloud benchmarks.

Related papers

• PointSea: Point Cloud Completion via Self-structure Augmentation [22.84159461045977]
  Point cloud completion is a fundamental yet not well-solved problem in 3D vision. We propose PointSea for global-to-local point cloud completion.
  arXiv  Detail & Related papers  (2025-02-24T11:07:00Z)
• Fully-Geometric Cross-Attention for Point Cloud Registration [51.865371511201765]
  Point cloud registration approaches often fail when the overlap between point clouds is low due to noisy point correspondences. This work introduces a novel cross-attention mechanism tailored for Transformer-based architectures that tackles this problem. We integrate the Gromov-Wasserstein distance into the cross-attention formulation to jointly compute distances between points across different point clouds. At the point level, we also devise a self-attention mechanism that aggregates the local geometric structure information into point features for fine matching.
  arXiv  Detail & Related papers  (2025-02-12T10:44:36Z)
• Bridging Domain Gap of Point Cloud Representations via Self-Supervised Geometric Augmentation [15.881442863961531]
  We introduce a novel scheme for induced geometric invariance of point cloud representations across domains. On one hand, a novel pretext task of predicting translation of distances of augmented samples is proposed to alleviate centroid shift of point clouds. On the other hand, we pioneer an integration of the relational self-supervised learning on geometrically-augmented point clouds.
  arXiv  Detail & Related papers  (2024-09-11T02:39:19Z)
• Boosting Cross-Domain Point Classification via Distilling Relational Priors from 2D Transformers [59.0181939916084]
  Traditional 3D networks mainly focus on local geometric details and ignore the topological structure between local geometries. We propose a novel Priors Distillation (RPD) method to extract priors from the well-trained transformers on massive images. Experiments on the PointDA-10 and the Sim-to-Real datasets verify that the proposed method consistently achieves the state-of-the-art performance of UDA for point cloud classification.
  arXiv  Detail & Related papers  (2024-07-26T06:29:09Z)
• Self-supervised 3D Point Cloud Completion via Multi-view Adversarial Learning [61.14132533712537]
  We propose MAL-SPC, a framework that effectively leverages both object-level and category-specific geometric similarities to complete missing structures. Our MAL-SPC does not require any 3D complete supervision and only necessitates a single partial point cloud for each object.
  arXiv  Detail & Related papers  (2024-07-13T06:53:39Z)
• PointeNet: A Lightweight Framework for Effective and Efficient Point Cloud Analysis [28.54939134635978]
  PointeNet is a network designed specifically for point cloud analysis. Our method demonstrates flexibility by seamlessly integrating with a classification/segmentation head or embedding into off-the-shelf 3D object detection networks. Experiments on object-level datasets, including ModelNet40, ScanObjectNN, ShapeNet KITTI, and the scene-level dataset KITTI, demonstrate the superior performance of PointeNet over state-of-the-art methods in point cloud analysis.
  arXiv  Detail & Related papers  (2023-12-20T03:34:48Z)
• Variational Relational Point Completion Network for Robust 3D Classification [59.80993960827833]
  Vari point cloud completion methods tend to generate global shape skeletons hence lack fine local details. This paper proposes a variational framework, point Completion Network (VRCNet) with two appealing properties. VRCNet shows great generalizability and robustness on real-world point cloud scans.
  arXiv  Detail & Related papers  (2023-04-18T17:03:20Z)
• MetaSets: Meta-Learning on Point Sets for Generalizable Representations [100.5981809166658]
  We study a new problem of 3D Domain Generalization (3DDG) with the goal to generalize the model to other unseen domains of point clouds without access to them in the training process. We propose to tackle this problem via MetaSets, which meta-learns point cloud representations from a group of classification tasks on carefully-designed transformed point sets. We design two benchmarks for Sim-to-Real transfer of 3D point clouds. Experimental results show that MetaSets outperforms existing 3D deep learning methods by large margins.
  arXiv  Detail & Related papers  (2022-04-15T03:24:39Z)
• IDEA-Net: Dynamic 3D Point Cloud Interpolation via Deep Embedding Alignment [58.8330387551499]
  We formulate the problem as estimation of point-wise trajectories (i.e., smooth curves) We propose IDEA-Net, an end-to-end deep learning framework, which disentangles the problem under the assistance of the explicitly learned temporal consistency. We demonstrate the effectiveness of our method on various point cloud sequences and observe large improvement over state-of-the-art methods both quantitatively and visually.
  arXiv  Detail & Related papers  (2022-03-22T10:14:08Z)
• DFC: Deep Feature Consistency for Robust Point Cloud Registration [0.4724825031148411]
  We present a novel learning-based alignment network for complex alignment scenes. We validate our approach on the 3DMatch dataset and the KITTI odometry dataset.
  arXiv  Detail & Related papers  (2021-11-15T08:27:21Z)
• PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers [81.71904691925428]
  We present a new method that reformulates point cloud completion as a set-to-set translation problem. We also design a new model, called PoinTr, that adopts a transformer encoder-decoder architecture for point cloud completion. Our method outperforms state-of-the-art methods by a large margin on both the new benchmarks and the existing ones.
  arXiv  Detail & Related papers  (2021-08-19T17:58:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.