Collaborative 3D Object Detection for Automatic Vehicle Systems via Learnable Communications

• URL: http://arxiv.org/abs/2205.11849v1
• Date: Tue, 24 May 2022 07:17:32 GMT
• Title: Collaborative 3D Object Detection for Automatic Vehicle Systems via Learnable Communications
• Authors: Junyong Wang, Yuan Zeng and Yi Gong
• Abstract summary: We propose a novel collaborative 3D object detection framework that consists of three components. Experiment results and bandwidth usage analysis demonstrate that our approach can save communication and computation costs.
• Score: 8.633120731620307
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Accurate detection of objects in 3D point clouds is a key problem in autonomous driving systems. Collaborative perception can incorporate information from spatially diverse sensors and provide significant benefits for improving the perception accuracy of autonomous driving systems. In this work, we consider that the autonomous vehicle uses local point cloud data and combines information from neighboring infrastructures through wireless links for cooperative 3D object detection. However, information sharing among vehicle and infrastructures in predefined communication schemes may result in communication congestion and/or bring limited performance improvement. To this end, we propose a novel collaborative 3D object detection framework that consists of three components: feature learning networks that map point clouds into feature maps; an efficient communication block that propagates compact and fine-grained query feature maps from vehicle to support infrastructures and optimizes attention weights between query and key to refine support feature maps; a region proposal network that fuses local feature maps and weighted support feature maps for 3D object detection. We evaluate the performance of the proposed framework using a synthetic cooperative dataset created in two complex driving scenarios: a roundabout and a T-junction. Experiment results and bandwidth usage analysis demonstrate that our approach can save communication and computation costs and significantly improve detection performance under different detection difficulties in all scenarios.

Related papers

• Neural Semantic Map-Learning for Autonomous Vehicles [85.8425492858912]
  We present a mapping system that fuses local submaps gathered from a fleet of vehicles at a central instance to produce a coherent map of the road environment. Our method jointly aligns and merges the noisy and incomplete local submaps using a scene-specific Neural Signed Distance Field. We leverage memory-efficient sparse feature-grids to scale to large areas and introduce a confidence score to model uncertainty in scene reconstruction.
  arXiv  Detail & Related papers  (2024-10-10T10:10:03Z)
• InScope: A New Real-world 3D Infrastructure-side Collaborative Perception Dataset for Open Traffic Scenarios [13.821143687548494]
  This paper introduces a new 3D infrastructure-side collaborative perception dataset, abbreviated as inscope. InScope encapsulates a 20-day capture duration with 303 tracking trajectories and 187,787 3D bounding boxes annotated by experts.
  arXiv  Detail & Related papers  (2024-07-31T13:11:14Z)
• Cross-Cluster Shifting for Efficient and Effective 3D Object Detection in Autonomous Driving [69.20604395205248]
  We present a new 3D point-based detector model, named Shift-SSD, for precise 3D object detection in autonomous driving. We introduce an intriguing Cross-Cluster Shifting operation to unleash the representation capacity of the point-based detector. We conduct extensive experiments on the KITTI, runtime, and nuScenes datasets, and the results demonstrate the state-of-the-art performance of Shift-SSD.
  arXiv  Detail & Related papers  (2024-03-10T10:36:32Z)
• AGO-Net: Association-Guided 3D Point Cloud Object Detection Network [86.10213302724085]
  We propose a novel 3D detection framework that associates intact features for objects via domain adaptation. We achieve new state-of-the-art performance on the KITTI 3D detection benchmark in both accuracy and speed.
  arXiv  Detail & Related papers  (2022-08-24T16:54:38Z)
• Cyber Mobility Mirror: Deep Learning-based Real-time 3D Object Perception and Reconstruction Using Roadside LiDAR [14.566471856473813]
  Cyber Mobility Mirror is a next-generation real-time traffic surveillance system for 3D object detection, classification, tracking, and reconstruction. Results from field tests demonstrate that our prototype system can provide satisfactory perception performance with 96.99% precision and 83.62% recall. High-fidelity real-time traffic conditions can be displayed on the GUI of the equipped vehicle with a frequency of 3-4 Hz.
  arXiv  Detail & Related papers  (2022-02-28T01:58:24Z)
• Achieving Real-Time LiDAR 3D Object Detection on a Mobile Device [53.323878851563414]
  We propose a compiler-aware unified framework incorporating network enhancement and pruning search with the reinforcement learning techniques. Specifically, a generator Recurrent Neural Network (RNN) is employed to provide the unified scheme for both network enhancement and pruning search automatically. The proposed framework achieves real-time 3D object detection on mobile devices with competitive detection performance.
  arXiv  Detail & Related papers  (2020-12-26T19:41:15Z)
• LiveMap: Real-Time Dynamic Map in Automotive Edge Computing [14.195521569220448]
  LiveMap is a real-time dynamic map that detects, matches, and tracks objects on the road with crowdsourcing data from connected vehicles in sub-second. We develop the control plane of LiveMap that allows adaptive offloading of vehicle computations. We implement LiveMap on a small-scale testbed and develop a large-scale network simulator.
  arXiv  Detail & Related papers  (2020-12-16T15:00:49Z)
• Fine-Grained Vehicle Perception via 3D Part-Guided Visual Data Augmentation [77.60050239225086]
  We propose an effective training data generation process by fitting a 3D car model with dynamic parts to vehicles in real images. Our approach is fully automatic without any human interaction. We present a multi-task network for VUS parsing and a multi-stream network for VHI parsing.
  arXiv  Detail & Related papers  (2020-12-15T03:03:38Z)
• CoFF: Cooperative Spatial Feature Fusion for 3D Object Detection on Autonomous Vehicles [20.333191597167847]
  CoFF achieves a significant improvement in terms of both detection precision and effective detection range for autonomous vehicles. Results show that CoFF achieves a significant improvement in terms of both detection precision and effective detection range for autonomous vehicles.
  arXiv  Detail & Related papers  (2020-09-24T22:51:50Z)
• Improving Point Cloud Semantic Segmentation by Learning 3D Object Detection [102.62963605429508]
  Point cloud semantic segmentation plays an essential role in autonomous driving. Current 3D semantic segmentation networks focus on convolutional architectures that perform great for well represented classes. We propose a novel Aware 3D Semantic Detection (DASS) framework that explicitly leverages localization features from an auxiliary 3D object detection task.
  arXiv  Detail & Related papers  (2020-09-22T14:17:40Z)
• Cooperative Perception with Deep Reinforcement Learning for Connected Vehicles [7.7003495898919265]
  We present a cooperative perception scheme with deep reinforcement learning to enhance the detection accuracy for the surrounding objects. Our scheme mitigates the network load in vehicular communication networks and enhances the communication reliability.
  arXiv  Detail & Related papers  (2020-04-23T01:44:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.