Cyber Mobility Mirror: Deep Learning-based Real-time 3D Object
Perception and Reconstruction Using Roadside LiDAR
- URL: http://arxiv.org/abs/2202.13505v1
- Date: Mon, 28 Feb 2022 01:58:24 GMT
- Title: Cyber Mobility Mirror: Deep Learning-based Real-time 3D Object
Perception and Reconstruction Using Roadside LiDAR
- Authors: Zhengwei Bai, Saswat Priyadarshi Nayak, Xuanpeng Zhao, Guoyuan Wu,
Matthew J. Barth, Xuewei Qi, Yongkang Liu, Kentaro Oguchi
- Abstract summary: Cyber Mobility Mirror is a next-generation real-time traffic surveillance system for 3D object detection, classification, tracking, and reconstruction.
Results from field tests demonstrate that our prototype system can provide satisfactory perception performance with 96.99% precision and 83.62% recall.
High-fidelity real-time traffic conditions can be displayed on the GUI of the equipped vehicle with a frequency of 3-4 Hz.
- Score: 14.566471856473813
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Enabling Cooperative Driving Automation (CDA) requires high-fidelity and
real-time perception information, which is available from onboard sensors or
vehicle-to-everything (V2X) communications. Nevertheless, the accessibility of
this information may suffer from the range and occlusion of perception or
limited penetration rates in connectivity. In this paper, we introduce the
prototype of Cyber Mobility Mirror (CMM), a next-generation real-time traffic
surveillance system for 3D object detection, classification, tracking, and
reconstruction, to provide CAVs with wide-range high-fidelity perception
information in a mixed traffic environment. The CMM system consists of six main
components: 1) the data pre-processor to retrieve and pre-process raw data from
the roadside LiDAR; 2) the 3D object detector to generate 3D bounding boxes
based on point cloud data; 3) the multi-objects tracker to endow unique IDs to
detected objects and estimate their dynamic states; 4) the global locator to
map positioning information from the LiDAR coordinate to geographic coordinate
using coordinate transformation; 5) the cloud-based communicator to transmit
perception information from roadside sensors to equipped vehicles; and 6) the
onboard advisor to reconstruct and display the real-time traffic conditions via
Graphical User Interface (GUI). In this study, a field-operational prototype
system is deployed at a real-world intersection, University Avenue and Iowa
Avenue in Riverside, California to assess the feasibility and performance of
our CMM system. Results from field tests demonstrate that our CMM prototype
system can provide satisfactory perception performance with 96.99% precision
and 83.62% recall. High-fidelity real-time traffic conditions (at the object
level) can be displayed on the GUI of the equipped vehicle with a frequency of
3-4 Hz.
Related papers
- InScope: A New Real-world 3D Infrastructure-side Collaborative Perception Dataset for Open Traffic Scenarios [13.821143687548494]
This paper introduces a new 3D infrastructure-side collaborative perception dataset, abbreviated as inscope.
InScope encapsulates a 20-day capture duration with 303 tracking trajectories and 187,787 3D bounding boxes annotated by experts.
arXiv Detail & Related papers (2024-07-31T13:11:14Z) - Empowering Urban Traffic Management: Elevated 3D LiDAR for Data Collection and Advanced Object Detection Analysis [4.831084635928491]
This paper presents a novel framework that transforms the detection and analysis of 3D objects in traffic scenarios by utilizing the power of elevated LiDAR sensors.
Due to the limitation in obtaining real-world traffic datasets, we utilize the simulator to generate 3D point cloud for specific scenarios.
arXiv Detail & Related papers (2024-05-21T21:12:09Z) - Cross-Cluster Shifting for Efficient and Effective 3D Object Detection
in Autonomous Driving [69.20604395205248]
We present a new 3D point-based detector model, named Shift-SSD, for precise 3D object detection in autonomous driving.
We introduce an intriguing Cross-Cluster Shifting operation to unleash the representation capacity of the point-based detector.
We conduct extensive experiments on the KITTI, runtime, and nuScenes datasets, and the results demonstrate the state-of-the-art performance of Shift-SSD.
arXiv Detail & Related papers (2024-03-10T10:36:32Z) - Unsupervised Domain Adaptation for Self-Driving from Past Traversal
Features [69.47588461101925]
We propose a method to adapt 3D object detectors to new driving environments.
Our approach enhances LiDAR-based detection models using spatial quantized historical features.
Experiments on real-world datasets demonstrate significant improvements.
arXiv Detail & Related papers (2023-09-21T15:00:31Z) - AGO-Net: Association-Guided 3D Point Cloud Object Detection Network [86.10213302724085]
We propose a novel 3D detection framework that associates intact features for objects via domain adaptation.
We achieve new state-of-the-art performance on the KITTI 3D detection benchmark in both accuracy and speed.
arXiv Detail & Related papers (2022-08-24T16:54:38Z) - Ret3D: Rethinking Object Relations for Efficient 3D Object Detection in
Driving Scenes [82.4186966781934]
We introduce a simple, efficient, and effective two-stage detector, termed as Ret3D.
At the core of Ret3D is the utilization of novel intra-frame and inter-frame relation modules.
With negligible extra overhead, Ret3D achieves the state-of-the-art performance.
arXiv Detail & Related papers (2022-08-18T03:48:58Z) - Real-Time And Robust 3D Object Detection with Roadside LiDARs [20.10416681832639]
We design a 3D object detection model that can detect traffic participants in roadside LiDARs in real-time.
Our model uses an existing 3D detector as a baseline and improves its accuracy.
We make a significant contribution with our LiDAR-based 3D detector that can be used for smart city applications.
arXiv Detail & Related papers (2022-07-11T21:33:42Z) - Collaborative 3D Object Detection for Automatic Vehicle Systems via
Learnable Communications [8.633120731620307]
We propose a novel collaborative 3D object detection framework that consists of three components.
Experiment results and bandwidth usage analysis demonstrate that our approach can save communication and computation costs.
arXiv Detail & Related papers (2022-05-24T07:17:32Z) - Cyber Mobility Mirror for Enabling Cooperative Driving Automation: A
Co-Simulation Platform [16.542137414609606]
Co-simulation platform can simulate both the real world with a high-fidelity sensor perception system and the cyber world with a real-time 3D reconstruction system.
Mirror-world simulator is responsible for reconstructing 3D objects and their trajectories from the perceived information.
Roadside LiDAR-based real-time vehicle detection and 3D reconstruction system is prototyped as a study case.
arXiv Detail & Related papers (2022-01-24T05:27:20Z) - Cycle and Semantic Consistent Adversarial Domain Adaptation for Reducing
Simulation-to-Real Domain Shift in LiDAR Bird's Eye View [110.83289076967895]
We present a BEV domain adaptation method based on CycleGAN that uses prior semantic classification in order to preserve the information of small objects of interest during the domain adaptation process.
The quality of the generated BEVs has been evaluated using a state-of-the-art 3D object detection framework at KITTI 3D Object Detection Benchmark.
arXiv Detail & Related papers (2021-04-22T12:47:37Z) - Fine-Grained Vehicle Perception via 3D Part-Guided Visual Data
Augmentation [77.60050239225086]
We propose an effective training data generation process by fitting a 3D car model with dynamic parts to vehicles in real images.
Our approach is fully automatic without any human interaction.
We present a multi-task network for VUS parsing and a multi-stream network for VHI parsing.
arXiv Detail & Related papers (2020-12-15T03:03:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.