SEED: A Simple and Effective 3D DETR in Point Clouds

• URL: http://arxiv.org/abs/2407.10749v1
• Date: Mon, 15 Jul 2024 14:21:07 GMT
• Title: SEED: A Simple and Effective 3D DETR in Point Clouds
• Authors: Zhe Liu, Jinghua Hou, Xiaoqing Ye, Tong Wang, Jingdong Wang, Xiang Bai,
• Abstract summary: We argue that the main challenges are challenging due to the high sparsity and uneven distribution of point clouds. We propose a simple and effective 3D DETR method (SEED) for detecting 3D objects from point clouds.
• Score: 72.74016394325675
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recently, detection transformers (DETRs) have gradually taken a dominant position in 2D detection thanks to their elegant framework. However, DETR-based detectors for 3D point clouds are still difficult to achieve satisfactory performance. We argue that the main challenges are twofold: 1) How to obtain the appropriate object queries is challenging due to the high sparsity and uneven distribution of point clouds; 2) How to implement an effective query interaction by exploiting the rich geometric structure of point clouds is not fully explored. To this end, we propose a simple and effective 3D DETR method (SEED) for detecting 3D objects from point clouds, which involves a dual query selection (DQS) module and a deformable grid attention (DGA) module. More concretely, to obtain appropriate queries, DQS first ensures a high recall to retain a large number of queries by the predicted confidence scores and then further picks out high-quality queries according to the estimated quality scores. DGA uniformly divides each reference box into grids as the reference points and then utilizes the predicted offsets to achieve a flexible receptive field, allowing the network to focus on relevant regions and capture more informative features. Extensive ablation studies on DQS and DGA demonstrate its effectiveness. Furthermore, our SEED achieves state-of-the-art detection performance on both the large-scale Waymo and nuScenes datasets, illustrating the superiority of our proposed method. The code is available at https://github.com/happinesslz/SEED

Related papers

• V-DETR: DETR with Vertex Relative Position Encoding for 3D Object Detection [73.37781484123536]
  We introduce a highly performant 3D object detector for point clouds using the DETR framework. To address the limitation, we introduce a novel 3D Relative Position (3DV-RPE) method. We show exceptional results on the challenging ScanNetV2 benchmark.
  arXiv  Detail & Related papers  (2023-08-08T17:14:14Z)
• Spatial-Temporal Graph Enhanced DETR Towards Multi-Frame 3D Object Detection [54.041049052843604]
  We present STEMD, a novel end-to-end framework that enhances the DETR-like paradigm for multi-frame 3D object detection. First, to model the inter-object spatial interaction and complex temporal dependencies, we introduce the spatial-temporal graph attention network. Finally, it poses a challenge for the network to distinguish between the positive query and other highly similar queries that are not the best match.
  arXiv  Detail & Related papers  (2023-07-01T13:53:14Z)
• PillarNeXt: Rethinking Network Designs for 3D Object Detection in LiDAR Point Clouds [29.15589024703907]
  In this paper, we revisit the local point aggregators from the perspective of allocating computational resources. We find that the simplest pillar based models perform surprisingly well considering both accuracy and latency. Our results challenge the common intuition that the detailed geometry modeling is essential to achieve high performance for 3D object detection.
  arXiv  Detail & Related papers  (2023-05-08T17:59:14Z)
• D2Q-DETR: Decoupling and Dynamic Queries for Oriented Object Detection with Transformers [14.488821968433834]
  We propose an end-to-end framework for oriented object detection. Our framework is based on DETR, with the box regression head replaced with a points prediction head. Experiments on the largest and challenging DOTA-v1.0 and DOTA-v1.5 datasets show that D2Q-DETR outperforms existing NMS-based and NMS-free oriented object detection methods.
  arXiv  Detail & Related papers  (2023-03-01T14:36:19Z)
• 3D Cascade RCNN: High Quality Object Detection in Point Clouds [122.42455210196262]
  We present 3D Cascade RCNN, which allocates multiple detectors based on the voxelized point clouds in a cascade paradigm. We validate the superiority of our proposed 3D Cascade RCNN, when comparing to state-of-the-art 3D object detection techniques.
  arXiv  Detail & Related papers  (2022-11-15T15:58:36Z)
• D-Align: Dual Query Co-attention Network for 3D Object Detection Based on Multi-frame Point Cloud Sequence [8.21339007493213]
  Conventional 3D object detectors detect objects using a set of points acquired over a fixed duration. Recent studies have shown that the performance of object detection can be further enhanced by utilizing point cloud sequences. We propose D-Align, which can effectively produce strong bird's-eye-view (BEV) features by aligning and aggregating the features obtained from a sequence of point sets.
  arXiv  Detail & Related papers  (2022-09-30T20:41:25Z)
• PC-RGNN: Point Cloud Completion and Graph Neural Network for 3D Object Detection [57.49788100647103]
  LiDAR-based 3D object detection is an important task for autonomous driving. Current approaches suffer from sparse and partial point clouds of distant and occluded objects. In this paper, we propose a novel two-stage approach, namely PC-RGNN, dealing with such challenges by two specific solutions.
  arXiv  Detail & Related papers  (2020-12-18T18:06:43Z)
• CLOCs: Camera-LiDAR Object Candidates Fusion for 3D Object Detection [13.986963122264633]
  We propose a novel Camera-LiDAR Object Candidates (CLOCs) fusion network. CLOCs fusion provides a low-complexity multi-modal fusion framework. We show that CLOCs ranks the highest among all the fusion-based methods in the official KITTI leaderboard.
  arXiv  Detail & Related papers  (2020-09-02T02:07:00Z)
• Cross-Modality 3D Object Detection [63.29935886648709]
  We present a novel two-stage multi-modal fusion network for 3D object detection. The whole architecture facilitates two-stage fusion. Our experiments on the KITTI dataset show that the proposed multi-stage fusion helps the network to learn better representations.
  arXiv  Detail & Related papers  (2020-08-16T11:01:20Z)
• 1st Place Solution for Waymo Open Dataset Challenge -- 3D Detection and Domain Adaptation [7.807118356899879]
  We propose a one-stage, anchor-free and NMS-free 3D point cloud object detector AFDet. AFDet serves as a strong baseline in our winning solution. We design stronger networks and enhance the point cloud data using densification and point painting.
  arXiv  Detail & Related papers  (2020-06-28T04:49:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.