End-to-End Autonomous Driving through V2X Cooperation

• URL: http://arxiv.org/abs/2404.00717v2
• Date: Sat, 20 Apr 2024 02:32:17 GMT
• Title: End-to-End Autonomous Driving through V2X Cooperation
• Authors: Haibao Yu, Wenxian Yang, Jiaru Zhong, Zhenwei Yang, Siqi Fan, Ping Luo, Zaiqing Nie,
• Abstract summary: We introduce UniV2X, a pioneering cooperative autonomous driving framework. UniV2X seamlessly integrates all key driving modules across diverse views into a unified network.
• Score: 23.44597411612664
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Cooperatively utilizing both ego-vehicle and infrastructure sensor data via V2X communication has emerged as a promising approach for advanced autonomous driving. However, current research mainly focuses on improving individual modules, rather than taking end-to-end learning to optimize final planning performance, resulting in underutilized data potential. In this paper, we introduce UniV2X, a pioneering cooperative autonomous driving framework that seamlessly integrates all key driving modules across diverse views into a unified network. We propose a sparse-dense hybrid data transmission and fusion mechanism for effective vehicle-infrastructure cooperation, offering three advantages: 1) Effective for simultaneously enhancing agent perception, online mapping, and occupancy prediction, ultimately improving planning performance. 2) Transmission-friendly for practical and limited communication conditions. 3) Reliable data fusion with interpretability of this hybrid data. We implement UniV2X, as well as reproducing several benchmark methods, on the challenging DAIR-V2X, the real-world cooperative driving dataset. Experimental results demonstrate the effectiveness of UniV2X in significantly enhancing planning performance, as well as all intermediate output performance. Code is at https://github.com/AIR-THU/UniV2X.

Related papers

• DiFSD: Ego-Centric Fully Sparse Paradigm with Uncertainty Denoising and Iterative Refinement for Efficient End-to-End Autonomous Driving [55.53171248839489]
  We propose an ego-centric fully sparse paradigm, named DiFSD, for end-to-end self-driving. Specifically, DiFSD mainly consists of sparse perception, hierarchical interaction and iterative motion planner. Experiments conducted on nuScenes dataset demonstrate the superior planning performance and great efficiency of DiFSD.
  arXiv  Detail & Related papers  (2024-09-15T15:55:24Z)
• CooPre: Cooperative Pretraining for V2X Cooperative Perception [47.00472259100765]
  We present a self-supervised learning method for V2X cooperative perception. We utilize the vast amount of unlabeled 3D V2X data to enhance the perception performance.
  arXiv  Detail & Related papers  (2024-08-20T23:39:26Z)
• Conformal Trajectory Prediction with Multi-View Data Integration in Cooperative Driving [4.628774934971078]
  Current research on trajectory prediction primarily relies on data collected by onboard sensors of an ego vehicle. We introduce V2INet, a novel trajectory prediction framework designed to model multi-view data by extending existing single-view models. Our results demonstrate superior performance in terms of Final Displacement Error (FDE) and Miss Rate (MR) using a single GPU.
  arXiv  Detail & Related papers  (2024-08-01T08:32:03Z)
• Hybrid-Generative Diffusion Models for Attack-Oriented Twin Migration in Vehicular Metaverses [58.264499654343226]
  Vehicle Twins (VTs) are digital twins that provide immersive virtual services for Vehicular Metaverse Users (VMUs) High mobility of vehicles, uneven deployment of edge servers, and potential security threats pose challenges to achieving efficient and reliable VT migrations. We propose a secure and reliable VT migration framework in vehicular metaverses.
  arXiv  Detail & Related papers  (2024-07-05T11:11:33Z)
• Unified End-to-End V2X Cooperative Autonomous Driving [21.631099800753795]
  UniE2EV2X is a V2X-integrated end-to-end autonomous driving system that consolidates key driving modules within a unified network. The framework employs a deformable attention-based data fusion strategy, effectively facilitating cooperation between vehicles and infrastructure. We implement the UniE2EV2X framework on the challenging DeepAccident, a simulation dataset designed for V2X cooperative driving.
  arXiv  Detail & Related papers  (2024-05-07T03:01:40Z)
• Towards Collaborative Autonomous Driving: Simulation Platform and End-to-End System [35.447617290190294]
  Vehicle-to-everything-aided autonomous driving (V2X-AD) has a huge potential to provide a safer driving solution. We present V2Xverse, a comprehensive simulation platform for collaborative autonomous driving. We introduce CoDriving, a novel end-to-end collaborative driving system.
  arXiv  Detail & Related papers  (2024-04-15T06:33:32Z)
• Learning Cooperative Trajectory Representations for Motion Forecasting [4.380073528690906]
  We propose a forecasting-oriented representation paradigm to utilize motion and interaction features from cooperative information. We present V2X-Graph, a representative framework to achieve interpretable and end-to-end trajectory feature fusion for cooperative motion forecasting. To further evaluate on vehicle-to-everything (V2X) scenario, we construct the first real-world V2X motion forecasting dataset V2X-Traj.
  arXiv  Detail & Related papers  (2023-11-01T08:53:05Z)
• FSD V2: Improving Fully Sparse 3D Object Detection with Virtual Voxels [57.05834683261658]
  We present FSDv2, an evolution that aims to simplify the previous FSDv1 while eliminating the inductive bias introduced by its handcrafted instance-level representation. We develop a suite of components to complement the virtual voxel concept, including a virtual voxel encoder, a virtual voxel mixer, and a virtual voxel assignment strategy.
  arXiv  Detail & Related papers  (2023-08-07T17:59:48Z)
• V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer [58.71845618090022]
  We build a holistic attention model, namely V2X-ViT, to fuse information across on-road agents. V2X-ViT consists of alternating layers of heterogeneous multi-agent self-attention and multi-scale window self-attention. To validate our approach, we create a large-scale V2X perception dataset.
  arXiv  Detail & Related papers  (2022-03-20T20:18:25Z)
• V2X-Sim: A Virtual Collaborative Perception Dataset for Autonomous Driving [26.961213523096948]
  Vehicle-to-everything (V2X) denotes the collaboration between a vehicle and any entity in its surrounding. We present the V2X-Sim dataset, the first public large-scale collaborative perception dataset in autonomous driving.
  arXiv  Detail & Related papers  (2022-02-17T05:14:02Z)
• Value Function is All You Need: A Unified Learning Framework for Ride Hailing Platforms [57.21078336887961]
  Large ride-hailing platforms, such as DiDi, Uber and Lyft, connect tens of thousands of vehicles in a city to millions of ride demands throughout the day. We propose a unified value-based dynamic learning framework (V1D3) for tackling both tasks.
  arXiv  Detail & Related papers  (2021-05-18T19:22:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.