Multi-View Attentive Contextualization for Multi-View 3D Object Detection

• URL: http://arxiv.org/abs/2405.12200v1
• Date: Mon, 20 May 2024 17:37:10 GMT
• Title: Multi-View Attentive Contextualization for Multi-View 3D Object Detection
• Authors: Xianpeng Liu, Ce Zheng, Ming Qian, Nan Xue, Chen Chen, Zhebin Zhang, Chen Li, Tianfu Wu,
• Abstract summary: We present Multi-View Attentive Contextualization (MvACon), a simple yet effective method for improving 2D-to-3D feature lifting in query-based 3D (MV3D) object detection. In experiments, the proposed MvACon is thoroughly tested on the nuScenes benchmark, using both the BEVFormer and its recent 3D deformable attention (DFA3D) variant, as well as the PETR.
• Score: 19.874148893464607
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present Multi-View Attentive Contextualization (MvACon), a simple yet effective method for improving 2D-to-3D feature lifting in query-based multi-view 3D (MV3D) object detection. Despite remarkable progress witnessed in the field of query-based MV3D object detection, prior art often suffers from either the lack of exploiting high-resolution 2D features in dense attention-based lifting, due to high computational costs, or from insufficiently dense grounding of 3D queries to multi-scale 2D features in sparse attention-based lifting. Our proposed MvACon hits the two birds with one stone using a representationally dense yet computationally sparse attentive feature contextualization scheme that is agnostic to specific 2D-to-3D feature lifting approaches. In experiments, the proposed MvACon is thoroughly tested on the nuScenes benchmark, using both the BEVFormer and its recent 3D deformable attention (DFA3D) variant, as well as the PETR, showing consistent detection performance improvement, especially in enhancing performance in location, orientation, and velocity prediction. It is also tested on the Waymo-mini benchmark using BEVFormer with similar improvement. We qualitatively and quantitatively show that global cluster-based contexts effectively encode dense scene-level contexts for MV3D object detection. The promising results of our proposed MvACon reinforces the adage in computer vision -- ``(contextualized) feature matters".

Related papers

• Towards Unified 3D Object Detection via Algorithm and Data Unification [70.27631528933482]
  We build the first unified multi-modal 3D object detection benchmark MM- Omni3D and extend the aforementioned monocular detector to its multi-modal version. We name the designed monocular and multi-modal detectors as UniMODE and MM-UniMODE, respectively.
  arXiv  Detail & Related papers  (2024-02-28T18:59:31Z)
• SOGDet: Semantic-Occupancy Guided Multi-view 3D Object Detection [19.75965521357068]
  We propose a novel approach called SOGDet (Semantic-Occupancy Guided Multi-view 3D Object Detection) to improve the accuracy of 3D object detection. Our results show that SOGDet consistently enhance the performance of three baseline methods in terms of nuScenes Detection Score (NDS) and mean Average Precision (mAP) This indicates that the combination of 3D object detection and 3D semantic occupancy leads to a more comprehensive perception of the 3D environment, thereby aiding build more robust autonomous driving systems.
  arXiv  Detail & Related papers  (2023-08-26T07:38:21Z)
• Fully Sparse Fusion for 3D Object Detection [69.32694845027927]
  Currently prevalent multimodal 3D detection methods are built upon LiDAR-based detectors that usually use dense Bird's-Eye-View feature maps. Fully sparse architecture is gaining attention as they are highly efficient in long-range perception. In this paper, we study how to effectively leverage image modality in the emerging fully sparse architecture.
  arXiv  Detail & Related papers  (2023-04-24T17:57:43Z)
• LiDAR-Based 3D Object Detection via Hybrid 2D Semantic Scene Generation [38.38852904444365]
  This paper proposes a novel scene representation that encodes both the semantics and geometry of the 3D environment in 2D. Our simple yet effective design can be easily integrated into most state-of-the-art 3D object detectors.
  arXiv  Detail & Related papers  (2023-04-04T04:05:56Z)
• CMR3D: Contextualized Multi-Stage Refinement for 3D Object Detection [57.44434974289945]
  We propose Contextualized Multi-Stage Refinement for 3D Object Detection (CMR3D) framework. Our framework takes a 3D scene as input and strives to explicitly integrate useful contextual information of the scene. In addition to 3D object detection, we investigate the effectiveness of our framework for the problem of 3D object counting.
  arXiv  Detail & Related papers  (2022-09-13T05:26:09Z)
• A Simple Baseline for Multi-Camera 3D Object Detection [94.63944826540491]
  3D object detection with surrounding cameras has been a promising direction for autonomous driving. We present SimMOD, a Simple baseline for Multi-camera Object Detection. We conduct extensive experiments on the 3D object detection benchmark of nuScenes to demonstrate the effectiveness of SimMOD.
  arXiv  Detail & Related papers  (2022-08-22T03:38:01Z)
• Improving 3D Object Detection with Channel-wise Transformer [58.668922561622466]
  We propose a two-stage 3D object detection framework (CT3D) with minimal hand-crafted design. CT3D simultaneously performs proposal-aware embedding and channel-wise context aggregation. It achieves the AP of 81.77% in the moderate car category on the KITTI test 3D detection benchmark.
  arXiv  Detail & Related papers  (2021-08-23T02:03:40Z)
• M3DSSD: Monocular 3D Single Stage Object Detector [82.25793227026443]
  We propose a Monocular 3D Single Stage object Detector (M3DSSD) with feature alignment and asymmetric non-local attention. The proposed M3DSSD achieves significantly better performance than the monocular 3D object detection methods on the KITTI dataset.
  arXiv  Detail & Related papers  (2021-03-24T13:09:11Z)
• IAFA: Instance-aware Feature Aggregation for 3D Object Detection from a Single Image [37.83574424518901]
  3D object detection from a single image is an important task in Autonomous Driving. We propose an instance-aware approach to aggregate useful information for improving the accuracy of 3D object detection.
  arXiv  Detail & Related papers  (2021-03-05T05:47:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.