PointTransformerX: Portable and Efficient 3D Point Cloud Processing without Sparse Algorithms

• URL: http://arxiv.org/abs/2604.24169v2
• Date: Wed, 29 Apr 2026 07:44:36 GMT
• Title: PointTransformerX: Portable and Efficient 3D Point Cloud Processing without Sparse Algorithms
• Abstract summary: PointTransformerX (PTX) is a PyTorch-native vision transformer backbone for 3D point clouds.<n>PTX removes all custom operators and external libraries while retaining competitive accuracy.
• Score: 0.10705399532413612
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: 3D point cloud perception remains tightly coupled to custom CUDA operators for spatial operations, limiting portability and efficiency on non-NVIDIA, AMD, and embedded hardware. We introduce PointTransformerX (PTX), a fully PyTorch-native vision transformer backbone for 3D point clouds, removing all custom CUDA operators and external libraries while retaining competitive accuracy. PTX introduces 3D-GS-RoPE, a rotary positional embedding that encodes 3D spatial relationships directly in self-attention without neighborhood construction, and further replaces sparse convolutional patch embedding with a linear projection. PTX explores inference-time scaling of attention windows to improve accuracy without retraining. With a redesigned feed-forward network, PTX achieves 98.7\% of PointTransformer V3's accuracy on ScanNet with 79.2\% fewer parameters and executing 1.6\times faster while requiring just 253 MB memory. PTX runs natively on NVIDIA GPUs, AMD GPUs (ROCm), and CPUs, providing an efficient and portable foundation for point cloud perception.

Related papers

• Token Adaptation via Side Graph Convolution for Efficient Fine-tuning of 3D Point Cloud Transformers [1.19658449368018]
  This paper proposes a novel PEFT algorithm called Side Token Adaptation on a neighborhood Graph (STAG) to achieve superior temporal and spatial efficiency.<n>STAG employs a graph convolutional side network operating in parallel with a frozen backbone Transformer to adapt tokens to downstream tasks.<n>We also present Point Cloud Classification 13 (PCC13), a new benchmark comprising diverse publicly available 3D point cloud datasets.
  arXiv  Detail & Related papers  (2025-02-19T22:58:56Z)
• PVTransformer: Point-to-Voxel Transformer for Scalable 3D Object Detection [36.04323550267339]
  3D object detectors for point clouds often rely on a pooling-based PointNet to encode sparse points into grid-like voxels or pillars. We propose PVTransformer: a transformer-based point-to-voxel architecture for 3D detection.
  arXiv  Detail & Related papers  (2024-05-05T04:44:41Z)
• ConDaFormer: Disassembled Transformer with Local Structure Enhancement for 3D Point Cloud Understanding [105.98609765389895]
  Transformers have been recently explored for 3D point cloud understanding. A large number of points, over 0.1 million, make the global self-attention infeasible for point cloud data. In this paper, we develop a new transformer block, named ConDaFormer.
  arXiv  Detail & Related papers  (2023-12-18T11:19:45Z)
• PVT-SSD: Single-Stage 3D Object Detector with Point-Voxel Transformer [75.2251801053839]
  We present a novel Point-Voxel Transformer for single-stage 3D detection (PVT-SSD) We propose a Point-Voxel Transformer (PVT) module that obtains long-range contexts in a cheap manner from voxels. The experiments on several autonomous driving benchmarks verify the effectiveness and efficiency of the proposed method.
  arXiv  Detail & Related papers  (2023-05-11T07:37:15Z)
• FastPillars: A Deployment-friendly Pillar-based 3D Detector [63.0697065653061]
  Existing BEV-based (i.e., Bird Eye View) detectors favor sparse convolutions (known as SPConv) to speed up training and inference. FastPillars delivers state-of-the-art accuracy on Open dataset with 1.8X speed up and 3.8 mAPH/L2 improvement over CenterPoint (SPConv-based)
  arXiv  Detail & Related papers  (2023-02-05T12:13:27Z)
• FlatFormer: Flattened Window Attention for Efficient Point Cloud Transformer [30.596658616831945]
  Transformer, as an alternative to CNN, has been proven effective in many modalities. This paper presents FlatFormer to close this latency gap by trading spatial proximity for better computational regularity.
  arXiv  Detail & Related papers  (2023-01-20T18:59:57Z)
• DSVT: Dynamic Sparse Voxel Transformer with Rotated Sets [95.84755169585492]
  We present Dynamic Sparse Voxel Transformer (DSVT), a single-stride window-based voxel Transformer backbone for outdoor 3D perception. Our model achieves state-of-the-art performance with a broad range of 3D perception tasks.
  arXiv  Detail & Related papers  (2023-01-15T09:31:58Z)
• TorchSparse: Efficient Point Cloud Inference Engine [24.541195361633523]
  We introduce TorchSparse, a high-performance point cloud inference engine. TorchSparse directly optimize the two bottlenecks of sparse convolution: irregular computation and data movement. It achieves 1.6x and 1.5x measured end-to-end speedup over the state-of-the-art MinkowskiEngine and SpConv, respectively.
  arXiv  Detail & Related papers  (2022-04-21T17:58:30Z)
• PiFeNet: Pillar-Feature Network for Real-Time 3D Pedestrian Detection from Point Cloud [64.12626752721766]
  We present PiFeNet, an efficient real-time 3D detector for pedestrian detection from point clouds. We address two challenges that 3D object detection frameworks encounter when detecting pedestrians: low of pillar features and small occupation areas of pedestrians in point clouds. Our approach is ranked 1st in KITTI pedestrian BEV and 3D leaderboards while running at 26 frames per second (FPS), and achieves state-of-the-art performance on Nuscenes detection benchmark.
  arXiv  Detail & Related papers  (2021-12-31T13:41:37Z)
• End-to-End Pseudo-LiDAR for Image-Based 3D Object Detection [62.34374949726333]
  Pseudo-LiDAR (PL) has led to a drastic reduction in the accuracy gap between methods based on LiDAR sensors and those based on cheap stereo cameras. PL combines state-of-the-art deep neural networks for 3D depth estimation with those for 3D object detection by converting 2D depth map outputs to 3D point cloud inputs. We introduce a new framework based on differentiable Change of Representation (CoR) modules that allow the entire PL pipeline to be trained end-to-end.
  arXiv  Detail & Related papers  (2020-04-07T02:18:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.