Spatial-Temporal Graph Enhanced DETR Towards Multi-Frame 3D Object Detection

• URL: http://arxiv.org/abs/2307.00347v4
• Date: Tue, 13 Aug 2024 03:22:50 GMT
• Title: Spatial-Temporal Graph Enhanced DETR Towards Multi-Frame 3D Object Detection
• Authors: Yifan Zhang, Zhiyu Zhu, Junhui Hou, Dapeng Wu,
• Abstract summary: We present STEMD, a novel end-to-end framework that enhances the DETR-like paradigm for multi-frame 3D object detection. First, to model the inter-object spatial interaction and complex temporal dependencies, we introduce the spatial-temporal graph attention network. Finally, it poses a challenge for the network to distinguish between the positive query and other highly similar queries that are not the best match.
• Score: 54.041049052843604
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The Detection Transformer (DETR) has revolutionized the design of CNN-based object detection systems, showcasing impressive performance. However, its potential in the domain of multi-frame 3D object detection remains largely unexplored. In this paper, we present STEMD, a novel end-to-end framework that enhances the DETR-like paradigm for multi-frame 3D object detection by addressing three key aspects specifically tailored for this task. First, to model the inter-object spatial interaction and complex temporal dependencies, we introduce the spatial-temporal graph attention network, which represents queries as nodes in a graph and enables effective modeling of object interactions within a social context. To solve the problem of missing hard cases in the proposed output of the encoder in the current frame, we incorporate the output of the previous frame to initialize the query input of the decoder. Finally, it poses a challenge for the network to distinguish between the positive query and other highly similar queries that are not the best match. And similar queries are insufficiently suppressed and turn into redundant prediction boxes. To address this issue, our proposed IoU regularization term encourages similar queries to be distinct during the refinement. Through extensive experiments, we demonstrate the effectiveness of our approach in handling challenging scenarios, while incurring only a minor additional computational overhead. The code is publicly available at https://github.com/Eaphan/STEMD.

Related papers

• A Modern Take on Visual Relationship Reasoning for Grasp Planning [10.543168383800532]
  We present a modern take on visual relational reasoning for grasp planning. We introduce D3GD, a novel testbed that includes bin picking scenes with up to 35 objects from 97 distinct categories. We also propose D3G, a new end-to-end transformer-based dependency graph generation model.
  arXiv  Detail & Related papers  (2024-09-03T16:30:48Z)
• Diff3DETR:Agent-based Diffusion Model for Semi-supervised 3D Object Detection [33.58208166717537]
  3D object detection is essential for understanding 3D scenes. Recent developments in semi-supervised methods seek to mitigate this problem by employing a teacher-student framework to generate pseudo-labels for unlabeled point clouds. We introduce an Agent-based Diffusion Model for Semi-supervised 3D Object Detection (Diff3DETR)
  arXiv  Detail & Related papers  (2024-08-01T05:04:22Z)
• SSGA-Net: Stepwise Spatial Global-local Aggregation Networks for for Autonomous Driving [27.731481134782577]
  Current models usually aggregate features from the neighboring frames to enhance the object representations for the task heads. These methods rely on the information from the future frames and suffer from high computational complexity. We introduce a stepwise spatial global-local aggregation network to solve these problems.
  arXiv  Detail & Related papers  (2024-05-29T08:12:51Z)
• PoIFusion: Multi-Modal 3D Object Detection via Fusion at Points of Interest [65.48057241587398]
  PoIFusion is a framework to fuse information of RGB images and LiDAR point clouds at the points of interest (PoIs) Our approach maintains the view of each modality and obtains multi-modal features by computation-friendly projection and computation. We conducted extensive experiments on nuScenes and Argoverse2 datasets to evaluate our approach.
  arXiv  Detail & Related papers  (2024-03-14T09:28:12Z)
• DETR4D: Direct Multi-View 3D Object Detection with Sparse Attention [50.11672196146829]
  3D object detection with surround-view images is an essential task for autonomous driving. We propose DETR4D, a Transformer-based framework that explores sparse attention and direct feature query for 3D object detection in multi-view images.
  arXiv  Detail & Related papers  (2022-12-15T14:18:47Z)
• 3D-QueryIS: A Query-based Framework for 3D Instance Segmentation [74.6998931386331]
  Previous methods for 3D instance segmentation often maintain inter-task dependencies and the tendency towards a lack of robustness. We propose a novel query-based method, termed as 3D-QueryIS, which is detector-free, semantic segmentation-free, and cluster-free. Our 3D-QueryIS is free from the accumulated errors caused by the inter-task dependencies.
  arXiv  Detail & Related papers  (2022-11-17T07:04:53Z)
• Efficient Person Search: An Anchor-Free Approach [86.45858994806471]
  Person search aims to simultaneously localize and identify a query person from realistic, uncropped images. To achieve this goal, state-of-the-art models typically add a re-id branch upon two-stage detectors like Faster R-CNN. In this work, we present an anchor-free approach to efficiently tackling this challenging task, by introducing the following dedicated designs.
  arXiv  Detail & Related papers  (2021-09-01T07:01:33Z)
• PC-RGNN: Point Cloud Completion and Graph Neural Network for 3D Object Detection [57.49788100647103]
  LiDAR-based 3D object detection is an important task for autonomous driving. Current approaches suffer from sparse and partial point clouds of distant and occluded objects. In this paper, we propose a novel two-stage approach, namely PC-RGNN, dealing with such challenges by two specific solutions.
  arXiv  Detail & Related papers  (2020-12-18T18:06:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.