BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detection

• URL: http://arxiv.org/abs/2206.10092v1
• Date: Tue, 21 Jun 2022 03:21:18 GMT
• Title: BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detection
• Authors: Yinhao Li, Zheng Ge, Guanyi Yu, Jinrong Yang, Zengran Wang, Yukang Shi, Jianjian Sun, Zeming Li
• Abstract summary: We propose a new 3D object detector with a trustworthy depth estimation, dubbed BEVDepth, for camera-based Bird's-Eye-View 3D object detection. BEVDepth achieves the new state-of-the-art 60.0% NDS on the challenging nuScenes test set.
• Score: 13.319949358652192
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this research, we propose a new 3D object detector with a trustworthy depth estimation, dubbed BEVDepth, for camera-based Bird's-Eye-View (BEV) 3D object detection. By a thorough analysis of recent approaches, we discover that the depth estimation is implicitly learned without camera information, making it the de-facto fake-depth for creating the following pseudo point cloud. BEVDepth gets explicit depth supervision utilizing encoded intrinsic and extrinsic parameters. A depth correction sub-network is further introduced to counteract projecting-induced disturbances in depth ground truth. To reduce the speed bottleneck while projecting features from image-view into BEV using estimated depth, a quick view-transform operation is also proposed. Besides, our BEVDepth can be easily extended with input from multi-frame. Without any bells and whistles, BEVDepth achieves the new state-of-the-art 60.0% NDS on the challenging nuScenes test set while maintaining high efficiency. For the first time, the performance gap between the camera and LiDAR is largely reduced within 10% NDS.

Related papers

• OPEN: Object-wise Position Embedding for Multi-view 3D Object Detection [102.0744303467713]
  We propose a new multi-view 3D object detector named OPEN. Our main idea is to effectively inject object-wise depth information into the network through our proposed object-wise position embedding. OPEN achieves a new state-of-the-art performance with 64.4% NDS and 56.7% mAP on the nuScenes test benchmark.
  arXiv  Detail & Related papers  (2024-07-15T14:29:15Z)
• Toward Accurate Camera-based 3D Object Detection via Cascade Depth Estimation and Calibration [20.82054596017465]
  Recent camera-based 3D object detection is limited by the precision of transforming from image to 3D feature spaces. This paper aims to address such a fundamental problem of camera-based 3D object detection: How to effectively learn depth information for accurate feature lifting and object localization.
  arXiv  Detail & Related papers  (2024-02-07T14:21:26Z)
• Instance-aware Multi-Camera 3D Object Detection with Structural Priors Mining and Self-Boosting Learning [93.71280187657831]
  Camera-based bird-eye-view (BEV) perception paradigm has made significant progress in the autonomous driving field. We propose IA-BEV, which integrates image-plane instance awareness into the depth estimation process within a BEV-based detector.
  arXiv  Detail & Related papers  (2023-12-13T09:24:42Z)
• OA-BEV: Bringing Object Awareness to Bird's-Eye-View Representation for Multi-Camera 3D Object Detection [78.38062015443195]
  OA-BEV is a network that can be plugged into the BEV-based 3D object detection framework. Our method achieves consistent improvements over the BEV-based baselines in terms of both average precision and nuScenes detection score.
  arXiv  Detail & Related papers  (2023-01-13T06:02:31Z)
• BEVDistill: Cross-Modal BEV Distillation for Multi-View 3D Object Detection [17.526914782562528]
  3D object detection from multiple image views is a challenging task for visual scene understanding. We propose textbfBEVDistill, a cross-modal BEV knowledge distillation framework for multi-view 3D object detection. Our best model achieves 59.4 NDS on the nuScenes test leaderboard, achieving new state-of-the-art in comparison with various image-based detectors.
  arXiv  Detail & Related papers  (2022-11-17T07:26:14Z)
• Boosting Monocular 3D Object Detection with Object-Centric Auxiliary Depth Supervision [13.593246617391266]
  We propose a method to boost the RGB image-based 3D detector by jointly training the detection network with a depth prediction loss analogous to the depth estimation task. Our novel object-centric depth prediction loss focuses on depth around foreground objects, which is important for 3D object detection. Our depth regression model is further trained to predict the uncertainty of depth to represent the 3D confidence of objects.
  arXiv  Detail & Related papers  (2022-10-29T11:32:28Z)
• Depth Estimation Matters Most: Improving Per-Object Depth Estimation for Monocular 3D Detection and Tracking [47.59619420444781]
  Approaches to monocular 3D perception including detection and tracking often yield inferior performance when compared to LiDAR-based techniques. We propose a multi-level fusion method that combines different representations (RGB and pseudo-LiDAR) and temporal information across multiple frames for objects (tracklets) to enhance per-object depth estimation.
  arXiv  Detail & Related papers  (2022-06-08T03:37:59Z)
• MDS-Net: A Multi-scale Depth Stratification Based Monocular 3D Object Detection Algorithm [4.958840734249869]
  This paper proposes a one-stage monocular 3D object detection algorithm based on multi-scale depth stratification. Experiments on the KITTI benchmark show that the MDS-Net outperforms the existing monocular 3D detection methods in 3D detection and BEV detection tasks.
  arXiv  Detail & Related papers  (2022-01-12T07:11:18Z)
• Self-Attention Dense Depth Estimation Network for Unrectified Video Sequences [6.821598757786515]
  LiDAR and radar sensors are the hardware solution for real-time depth estimation. Deep learning based self-supervised depth estimation methods have shown promising results. We propose a self-attention based depth and ego-motion network for unrectified images.
  arXiv  Detail & Related papers  (2020-05-28T21:53:53Z)
• Lightweight Multi-View 3D Pose Estimation through Camera-Disentangled Representation [57.11299763566534]
  We present a solution to recover 3D pose from multi-view images captured with spatially calibrated cameras. We exploit 3D geometry to fuse input images into a unified latent representation of pose, which is disentangled from camera view-points. Our architecture then conditions the learned representation on camera projection operators to produce accurate per-view 2d detections.
  arXiv  Detail & Related papers  (2020-04-05T12:52:29Z)
• D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometry [57.5549733585324]
  D3VO is a novel framework for monocular visual odometry that exploits deep networks on three levels -- deep depth, pose and uncertainty estimation. We first propose a novel self-supervised monocular depth estimation network trained on stereo videos without any external supervision. We model the photometric uncertainties of pixels on the input images, which improves the depth estimation accuracy.
  arXiv  Detail & Related papers  (2020-03-02T17:47:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.