PointConvFormer: Revenge of the Point-based Convolution

• URL: http://arxiv.org/abs/2208.02879v3
• Date: Wed, 10 May 2023 21:43:09 GMT
• Title: PointConvFormer: Revenge of the Point-based Convolution
• Authors: Wenxuan Wu, Li Fuxin, Qi Shan
• Abstract summary: We introduce PointConvFormer, a novel building block for point cloud based deep network architectures. Inspired by generalization theory, PointConvFormer combines ideas from point convolution, where filter weights are only based on relative position, and Transformers which utilize feature-based attention. Our results show that PointConvFormer offers a better accuracy-speed tradeoff than classic convolutions, regular transformers, and voxelized sparse convolution approaches.
• Score: 7.539787913497268
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce PointConvFormer, a novel building block for point cloud based deep network architectures. Inspired by generalization theory, PointConvFormer combines ideas from point convolution, where filter weights are only based on relative position, and Transformers which utilize feature-based attention. In PointConvFormer, attention computed from feature difference between points in the neighborhood is used to modify the convolutional weights at each point. Hence, we preserved the invariances from point convolution, whereas attention helps to select relevant points in the neighborhood for convolution. PointConvFormer is suitable for multiple tasks that require details at the point level, such as segmentation and scene flow estimation tasks. We experiment on both tasks with multiple datasets including ScanNet, SemanticKitti, FlyingThings3D and KITTI. Our results show that PointConvFormer offers a better accuracy-speed tradeoff than classic convolutions, regular transformers, and voxelized sparse convolution approaches. Visualizations show that PointConvFormer performs similarly to convolution on flat areas, whereas the neighborhood selection effect is stronger on object boundaries, showing that it has got the best of both worlds.

Related papers

• ConDaFormer: Disassembled Transformer with Local Structure Enhancement for 3D Point Cloud Understanding [105.98609765389895]
  Transformers have been recently explored for 3D point cloud understanding. A large number of points, over 0.1 million, make the global self-attention infeasible for point cloud data. In this paper, we develop a new transformer block, named ConDaFormer.
  arXiv  Detail & Related papers  (2023-12-18T11:19:45Z)
• Point Cloud Classification Using Content-based Transformer via Clustering in Feature Space [25.57569871876213]
  We propose a point content-based Transformer architecture, called PointConT for short. It exploits the locality of points in the feature space (content-based), which clusters the sampled points with similar features into the same class and computes the self-attention within each class. We also introduce an Inception feature aggregator for point cloud classification, which uses parallel structures to aggregate high-frequency and low-frequency information in each branch separately.
  arXiv  Detail & Related papers  (2023-03-08T14:11:05Z)
• Stratified Transformer for 3D Point Cloud Segmentation [89.9698499437732]
  Stratified Transformer is able to capture long-range contexts and demonstrates strong generalization ability and high performance. To combat the challenges posed by irregular point arrangements, we propose first-layer point embedding to aggregate local information. Experiments demonstrate the effectiveness and superiority of our method on S3DIS, ScanNetv2 and ShapeNetPart datasets.
  arXiv  Detail & Related papers  (2022-03-28T05:35:16Z)
• Projection-based Point Convolution for Efficient Point Cloud Segmentation [24.375383511061955]
  Projection-based Point Convolution (PPConv) is a point convolutional module that uses 2D convolutions and multi-layer perceptrons (MLPs) as its components. PPConv achieves superior efficiency compared to state-of-the-art methods, even with a simple architecture based on PointNet++.
  arXiv  Detail & Related papers  (2022-02-04T06:22:33Z)
• Adaptive Graph Convolution for Point Cloud Analysis [25.175406613705274]
  We propose Adaptive Graph Convolution (AdaptConv) which generates adaptive kernels for points according to their dynamically learned features. Our method outperforms state-of-the-art point cloud classification and segmentation approaches on several benchmark datasets.
  arXiv  Detail & Related papers  (2021-08-18T08:38:52Z)
• SCTN: Sparse Convolution-Transformer Network for Scene Flow Estimation [71.2856098776959]
  Estimating 3D motions for point clouds is challenging, since a point cloud is unordered and its density is significantly non-uniform. We propose a novel architecture named Sparse Convolution-Transformer Network (SCTN) that equips the sparse convolution with the transformer. We show that the learned relation-based contextual information is rich and helpful for matching corresponding points, benefiting scene flow estimation.
  arXiv  Detail & Related papers  (2021-05-10T15:16:14Z)
• The Devils in the Point Clouds: Studying the Robustness of Point Cloud Convolutions [15.997907568429177]
  This paper investigates different variants of PointConv, a convolution network on point clouds, to examine their robustness to input scale and rotation changes. We derive a novel viewpoint-invariant descriptor by utilizing 3D geometric properties as the input to PointConv. Experiments are conducted on the 2D MNIST & CIFAR-10 datasets as well as the 3D Semantic KITTI & ScanNet dataset.
  arXiv  Detail & Related papers  (2021-01-19T19:32:38Z)
• Global Context Aware Convolutions for 3D Point Cloud Understanding [32.953907994511376]
  We propose a novel convolution operator that enhances feature distinction by integrating global context information from the input point cloud to the convolution. A convolution can then be performed to transform the points and anchor features into final rotation-invariant features.
  arXiv  Detail & Related papers  (2020-08-07T04:33:27Z)
• Point-Set Anchors for Object Detection, Instance Segmentation and Pose Estimation [85.96410825961966]
  We argue that the image features extracted at a central point contain limited information for predicting distant keypoints or bounding box boundaries. To facilitate inference, we propose to instead perform regression from a set of points placed at more advantageous positions. We apply this proposed framework, called Point-Set Anchors, to object detection, instance segmentation, and human pose estimation.
  arXiv  Detail & Related papers  (2020-07-06T15:59:56Z)
• FKAConv: Feature-Kernel Alignment for Point Cloud Convolution [75.85619090748939]
  We provide a formulation to relate and analyze a number of point convolution methods. We also propose our own convolution variant, that separates the estimation of geometry-less kernel weights. We show competitive results on classification and semantic segmentation benchmarks.
  arXiv  Detail & Related papers  (2020-04-09T10:12:45Z)
• FPConv: Learning Local Flattening for Point Convolution [64.01196188303483]
  We introduce FPConv, a novel surface-style convolution operator designed for 3D point cloud analysis. Unlike previous methods, FPConv doesn't require transforming to intermediate representation like 3D grid or graph. FPConv can be easily integrated into various network architectures for tasks like 3D object classification and 3D scene segmentation.
  arXiv  Detail & Related papers  (2020-02-25T07:15:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.