TUMTraf V2X Cooperative Perception Dataset
- URL: http://arxiv.org/abs/2403.01316v1
- Date: Sat, 2 Mar 2024 21:29:04 GMT
- Title: TUMTraf V2X Cooperative Perception Dataset
- Authors: Walter Zimmer, Gerhard Arya Wardana, Suren Sritharan, Xingcheng Zhou,
Rui Song, Alois C. Knoll
- Abstract summary: We propose CoopDet3D, a cooperative multi-modal fusion model, and TUMTraf-V2X, a perception dataset.
Our dataset contains 2,000 labeled point clouds and 5,000 labeled images from five roadside and four onboard sensors.
We show that our CoopDet3D camera-LiDAR fusion model achieves an increase of +14.36 3D mAP compared to a vehicle camera-LiDAR fusion model.
- Score: 20.907021313266128
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Cooperative perception offers several benefits for enhancing the capabilities
of autonomous vehicles and improving road safety. Using roadside sensors in
addition to onboard sensors increases reliability and extends the sensor range.
External sensors offer higher situational awareness for automated vehicles and
prevent occlusions. We propose CoopDet3D, a cooperative multi-modal fusion
model, and TUMTraf-V2X, a perception dataset, for the cooperative 3D object
detection and tracking task. Our dataset contains 2,000 labeled point clouds
and 5,000 labeled images from five roadside and four onboard sensors. It
includes 30k 3D boxes with track IDs and precise GPS and IMU data. We labeled
eight categories and covered occlusion scenarios with challenging driving
maneuvers, like traffic violations, near-miss events, overtaking, and U-turns.
Through multiple experiments, we show that our CoopDet3D camera-LiDAR fusion
model achieves an increase of +14.36 3D mAP compared to a vehicle camera-LiDAR
fusion model. Finally, we make our dataset, model, labeling tool, and dev-kit
publicly available on our website:
https://tum-traffic-dataset.github.io/tumtraf-v2x.
Related papers
- RoboSense: Large-scale Dataset and Benchmark for Multi-sensor Low-speed Autonomous Driving [62.5830455357187]
In this paper, we construct a multimodal data collection platform based on 3 main types of sensors (Camera, LiDAR and Fisheye)
A large-scale multi-sensor dataset is built, named RoboSense, to facilitate near-field scene understanding.
RoboSense contains more than 133K synchronized data with 1.4M 3D bounding box and IDs in the full $360circ$ view, forming 216K trajectories across 7.6K temporal sequences.
arXiv Detail & Related papers (2024-08-28T03:17:40Z) - HoloVIC: Large-scale Dataset and Benchmark for Multi-Sensor Holographic Intersection and Vehicle-Infrastructure Cooperative [23.293162454592544]
We constructed holographic intersections with various layouts to build a large-scale multi-sensor holographic vehicle-infrastructure cooperation dataset, called HoloVIC.
Our dataset includes 3 different types of sensors (Camera, Lidar, Fisheye) and employs 4 sensor-s based on the different intersections.
arXiv Detail & Related papers (2024-03-05T04:08:19Z) - A9 Intersection Dataset: All You Need for Urban 3D Camera-LiDAR Roadside
Perception [20.10416681832639]
A9 Intersection dataset consists of labeled LiDAR point clouds and synchronized camera images.
Our dataset consists of 4.8k images and point clouds with more than 57.4k manually labeled 3D boxes.
arXiv Detail & Related papers (2023-06-15T16:39:51Z) - Multi-Modal 3D Object Detection by Box Matching [109.43430123791684]
We propose a novel Fusion network by Box Matching (FBMNet) for multi-modal 3D detection.
With the learned assignments between 3D and 2D object proposals, the fusion for detection can be effectively performed by combing their ROI features.
arXiv Detail & Related papers (2023-05-12T18:08:51Z) - V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle
Cooperative Perception [49.7212681947463]
Vehicle-to-Vehicle (V2V) cooperative perception system has great potential to revolutionize the autonomous driving industry.
We present V2V4Real, the first large-scale real-world multi-modal dataset for V2V perception.
Our dataset covers a driving area of 410 km, comprising 20K LiDAR frames, 40K RGB frames, 240K annotated 3D bounding boxes for 5 classes, and HDMaps.
arXiv Detail & Related papers (2023-03-14T02:49:20Z) - aiMotive Dataset: A Multimodal Dataset for Robust Autonomous Driving
with Long-Range Perception [0.0]
This dataset consists of 176 scenes with synchronized and calibrated LiDAR, camera, and radar sensors covering a 360-degree field of view.
The collected data was captured in highway, urban, and suburban areas during daytime, night, and rain.
We trained unimodal and multimodal baseline models for 3D object detection.
arXiv Detail & Related papers (2022-11-17T10:19:59Z) - Benchmarking the Robustness of LiDAR-Camera Fusion for 3D Object
Detection [58.81316192862618]
Two critical sensors for 3D perception in autonomous driving are the camera and the LiDAR.
fusing these two modalities can significantly boost the performance of 3D perception models.
We benchmark the state-of-the-art fusion methods for the first time.
arXiv Detail & Related papers (2022-05-30T09:35:37Z) - Rope3D: TheRoadside Perception Dataset for Autonomous Driving and
Monocular 3D Object Detection Task [48.555440807415664]
We present the first high-diversity challenging Roadside Perception 3D dataset- Rope3D from a novel view.
The dataset consists of 50k images and over 1.5M 3D objects in various scenes.
We propose to leverage the geometry constraint to solve the inherent ambiguities caused by various sensors, viewpoints.
arXiv Detail & Related papers (2022-03-25T12:13:23Z) - One Million Scenes for Autonomous Driving: ONCE Dataset [91.94189514073354]
We introduce the ONCE dataset for 3D object detection in the autonomous driving scenario.
The data is selected from 144 driving hours, which is 20x longer than the largest 3D autonomous driving dataset available.
We reproduce and evaluate a variety of self-supervised and semi-supervised methods on the ONCE dataset.
arXiv Detail & Related papers (2021-06-21T12:28:08Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.