CMU-Drive and V2V-VLA: Cooperative Multi-agent Unified Driving with Reasoning Benchmark and Vehicle-to-Vehicle Vision-Language-Action Models

• URL: http://arxiv.org/abs/2608.07621v1
• Date: Fri, 07 Aug 2026 09:00:39 GMT
• Title: CMU-Drive and V2V-VLA: Cooperative Multi-agent Unified Driving with Reasoning Benchmark and Vehicle-to-Vehicle Vision-Language-Action Models
• Abstract summary: We present Cooperative Multi-agent Unified Driving with Reasoning (CMU-Drive), a closed-loop end-to-end benchmark for evaluating cooperative autonomous driving.<n>We propose Vehicle-to-Vehicle Vision-Language-Action (V2V-VLA), a cooperative VLA model that integrates cooperative driving into a single forward pass.
• Score: 4.009780255195637
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Vision-Language-Action (VLA) models have recently achieved impressive performance for end-to-end autonomous driving, yet existing approaches are primarily designed for an individual single autonomous driving agent with limited support for cooperative perception, reasoning, and planning. We present Cooperative Multi-agent Unified Driving with Reasoning (CMU-Drive), a closed-loop end-to-end benchmark for evaluating cooperative autonomous driving with multiple connected autonomous vehicles (CAVs) operating in safety-critical driving scenarios with background traffic participants. We further propose Vehicle-to-Vehicle Vision-Language-Action (V2V-VLA), a cooperative VLA model that integrates cooperative driving into a single forward pass by jointly generating driving actions, future waypoints, language reasoning, and communication policies. Experiments on CMU-Drive establish the first benchmark and baseline for cooperative VLA driving and provide a foundation for future research on multi-agent, closed-loop, end-to-end cooperative autonomous driving. Our code, benchmark, and model checkpoint will be publicly released to facilitate open-source research.

Related papers

• A Collaborative Multi-Modality Interaction for VLA-based End-to-End Autonomous Driving [96.81243316566764]
  Vision-Language-Action (VLA) models have emerged as a powerful paradigm for end-to-end autonomous driving.<n>We propose a robust VLA-based end-to-end autonomous driving system that combines multi-modality interaction with multi-trajectory planning and optimization.
  arXiv  Detail & Related papers  (2026-08-21T09:06:16Z)
• Multi-Agent Embodied Autonomous Driving: From V2X Information Exchange to Shared World Models [16.06080006560829]
  Autonomous driving is shifting from isolated vehicle intelligence toward multi-agent embodied systems that share perception, infer intent, and coordinate action under uncertainty.<n>We review more than 380 publications spanning vehicle-to-everything (V2X) communication, collaborative perception, inter-agent cognition, cooperative planning, end-to-end cooperative driving, and simulation and data engines for closed-loop validation.
  arXiv  Detail & Related papers  (2026-06-11T19:10:13Z)
• A Human-Oriented Cooperative Driving Approach: Integrating Driving Intention, State, and Conflict [50.983743505985956]
  We propose a Human-Oriented Cooperative Driving (HOCD) approach to establish more natural and effective human-vehicle interaction.<n>In implementation, we take both tactical and operational levels into account to ensure seamless human-vehicle cooperation.<n>Compared to other cooperative driving approaches, the proposed HOCD approach significantly enhances driving performance and mitigates human-machine conflict.
  arXiv  Detail & Related papers  (2025-12-29T05:51:00Z)
• ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving [49.07731497951963]
  ReCogDrive is a novel Reinforced Cognitive framework for end-to-end autonomous driving.<n>We introduce a hierarchical data pipeline that mimics the sequential cognitive process of human drivers.<n>We then address the language-action mismatch by injecting the VLM's learned driving priors into a diffusion planner.
  arXiv  Detail & Related papers  (2025-06-09T03:14:04Z)
• Towards Natural Language Communication for Cooperative Autonomous Driving via Self-Play [70.70505035012462]
  Using natural language as a vehicle-to-vehicle (V2V) communication protocol offers the potential for autonomous vehicles to drive cooperatively.<n>This paper introduces a novel method, LLM+Debrief, to learn a message generation and high-level decision-making policy for autonomous vehicles.<n>Our experimental results demonstrate that LLM+Debrief is more effective at generating meaningful and human-understandable natural language messages.
  arXiv  Detail & Related papers  (2025-05-23T19:40:09Z)
• CoLMDriver: LLM-based Negotiation Benefits Cooperative Autonomous Driving [28.101283061232472]
  Vehicle-to-vehicle (V2V) cooperative autonomous driving holds great promise for improving safety.<n>Traditional cooperative methods are constrained by rigid collaboration protocols and limited generalization to unseen interactive scenarios.<n>We propose CoLMDriver, the first full-pipeline LLM-based cooperative driving system.
  arXiv  Detail & Related papers  (2025-03-11T17:58:42Z)
• V2V-LLM: Vehicle-to-Vehicle Cooperative Autonomous Driving with Multi-Modal Large Language Models [31.537045261401666]
  We propose a novel problem setting that integrates a Multi-Modal Large Language Model into cooperative autonomous driving.<n>We also propose our baseline method Vehicle-to-Vehicle Multi-Modal Large Language Model (V2V-LLM)<n> Experimental results show that our proposed V2V-LLM can be a promising unified model architecture for performing various tasks in cooperative autonomous driving.
  arXiv  Detail & Related papers  (2025-02-14T08:05:41Z)
• WHALES: A Multi-Agent Scheduling Dataset for Enhanced Cooperation in Autonomous Driving [14.977628973132099]
  We introduce WHALES, the first large-scale V2X dataset explicitly designed to benchmark communication-aware agent scheduling and scalable cooperative perception.<n>WHALES features an average of 8.4 cooperative agents per scene and 2.01 million annotated 3D objects across diverse traffic scenarios.<n>It incorporates detailed communication metadata to emulate real-world communication bottlenecks, enabling rigorous evaluation of scheduling strategies.
  arXiv  Detail & Related papers  (2024-11-20T14:12:34Z)
• Towards Interactive and Learnable Cooperative Driving Automation: a Large Language Model-Driven Decision-Making Framework [87.7482313774741]
  Connected Autonomous Vehicles (CAVs) have begun to open road testing around the world, but their safety and efficiency performance in complex scenarios is still not satisfactory.<n>This paper proposes CoDrivingLLM, an interactive and learnable LLM-driven cooperative driving framework.
  arXiv  Detail & Related papers  (2024-09-19T14:36:00Z)
• COOPERNAUT: End-to-End Driving with Cooperative Perception for Networked Vehicles [54.61668577827041]
  We introduce COOPERNAUT, an end-to-end learning model that uses cross-vehicle perception for vision-based cooperative driving. Our experiments on AutoCastSim suggest that our cooperative perception driving models lead to a 40% improvement in average success rate.
  arXiv  Detail & Related papers  (2022-05-04T17:55:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.