Learning High-resolution Vector Representation from Multi-Camera Images for 3D Object Detection

• URL: http://arxiv.org/abs/2407.15354v1
• Date: Mon, 22 Jul 2024 03:51:18 GMT
• Title: Learning High-resolution Vector Representation from Multi-Camera Images for 3D Object Detection
• Abstract summary: We present a new camera-based 3D object detector with high-resolution vector representation: VectorFormer. The presented high-resolution vector representation is combined with the lower-resolution BEV representation to efficiently exploit 3D geometry from multi-camera images at a high resolution through our two novel modules: vector scattering and gathering.
• Score: 60.11588003015481
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The Bird's-Eye-View (BEV) representation is a critical factor that directly impacts the 3D object detection performance, but the traditional BEV grid representation induces quadratic computational cost as the spatial resolution grows. To address this limitation, we present a new camera-based 3D object detector with high-resolution vector representation: VectorFormer. The presented high-resolution vector representation is combined with the lower-resolution BEV representation to efficiently exploit 3D geometry from multi-camera images at a high resolution through our two novel modules: vector scattering and gathering. To this end, the learned vector representation with richer scene contexts can serve as the decoding query for final predictions. We conduct extensive experiments on the nuScenes dataset and demonstrate state-of-the-art performance in NDS and inference time. Furthermore, we investigate query-BEV-based methods incorporated with our proposed vector representation and observe a consistent performance improvement.

Related papers

• SparseVoxFormer: Sparse Voxel-based Transformer for Multi-modal 3D Object Detection [12.941263635455915]
  Most previous 3D object detection methods utilize the Bird's Eye View (BEV) space for intermediate feature representation. This paper focuses on the sparse nature of LiDAR point cloud data. We introduce a novel sparse voxel-based transformer network for 3D object detection, dubbed as SparseVoxFormer.
  arXiv  Detail & Related papers  (2025-03-11T06:52:25Z)
• EVT: Efficient View Transformation for Multi-Modal 3D Object Detection [2.9848894641223302]
  We propose a novel 3D object detector via efficient view transformation (EVT) EVT uses Adaptive Sampling and Adaptive Projection (ASAP) to generate 3D sampling points and adaptive kernels. It is designed to effectively utilize the obtained multi-modal BEV features within the transformer decoder.
  arXiv  Detail & Related papers  (2024-11-16T06:11:10Z)
• GeoBEV: Learning Geometric BEV Representation for Multi-view 3D Object Detection [36.245654685143016]
  Bird's-Eye-View (BEV) representation has emerged as a mainstream paradigm for multi-view 3D object detection. Existing methods overlook the geometric quality of BEV representation, leaving it in a low-resolution state.
  arXiv  Detail & Related papers  (2024-09-03T11:57:36Z)
• Instance-aware Multi-Camera 3D Object Detection with Structural Priors Mining and Self-Boosting Learning [93.71280187657831]
  Camera-based bird-eye-view (BEV) perception paradigm has made significant progress in the autonomous driving field. We propose IA-BEV, which integrates image-plane instance awareness into the depth estimation process within a BEV-based detector.
  arXiv  Detail & Related papers  (2023-12-13T09:24:42Z)
• OA-BEV: Bringing Object Awareness to Bird's-Eye-View Representation for Multi-Camera 3D Object Detection [78.38062015443195]
  OA-BEV is a network that can be plugged into the BEV-based 3D object detection framework. Our method achieves consistent improvements over the BEV-based baselines in terms of both average precision and nuScenes detection score.
  arXiv  Detail & Related papers  (2023-01-13T06:02:31Z)
• DETR4D: Direct Multi-View 3D Object Detection with Sparse Attention [50.11672196146829]
  3D object detection with surround-view images is an essential task for autonomous driving. We propose DETR4D, a Transformer-based framework that explores sparse attention and direct feature query for 3D object detection in multi-view images.
  arXiv  Detail & Related papers  (2022-12-15T14:18:47Z)
• RangeRCNN: Towards Fast and Accurate 3D Object Detection with Range Image Representation [35.6155506566957]
  RangeRCNN is a novel and effective 3D object detection framework based on the range image representation. In this paper, we utilize the dilated residual block (DRB) to better adapt different object scales and obtain a more flexible receptive field. Experiments show that RangeRCNN achieves state-of-the-art performance on the KITTI dataset and the Open dataset.
  arXiv  Detail & Related papers  (2020-09-01T03:28:13Z)
• Lightweight Multi-View 3D Pose Estimation through Camera-Disentangled Representation [57.11299763566534]
  We present a solution to recover 3D pose from multi-view images captured with spatially calibrated cameras. We exploit 3D geometry to fuse input images into a unified latent representation of pose, which is disentangled from camera view-points. Our architecture then conditions the learned representation on camera projection operators to produce accurate per-view 2d detections.
  arXiv  Detail & Related papers  (2020-04-05T12:52:29Z)
• BirdNet+: End-to-End 3D Object Detection in LiDAR Bird's Eye View [117.44028458220427]
  On-board 3D object detection in autonomous vehicles often relies on geometry information captured by LiDAR devices. We present a fully end-to-end 3D object detection framework that can infer oriented 3D boxes solely from BEV images.
  arXiv  Detail & Related papers  (2020-03-09T15:08:40Z)
• Monocular 3D Object Detection with Decoupled Structured Polygon Estimation and Height-Guided Depth Estimation [41.29145717658494]
  This paper proposes a novel unified framework which decomposes the detection problem into a structured polygon prediction task and a depth recovery task. Compared to the widely-used 3D bounding box proposals, it is shown to be a better representation for 3D detection. Experiments are conducted on the challenging KITTI benchmark, in which our method achieves state-of-the-art detection accuracy.
  arXiv  Detail & Related papers  (2020-02-05T03:25:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.