CoLMDriver: LLM-based Negotiation Benefits Cooperative Autonomous Driving

• URL: http://arxiv.org/abs/2503.08683v1
• Date: Tue, 11 Mar 2025 17:58:42 GMT
• Title: CoLMDriver: LLM-based Negotiation Benefits Cooperative Autonomous Driving
• Authors: Changxing Liu, Genjia Liu, Zijun Wang, Jinchang Yang, Siheng Chen,
• Abstract summary: Vehicle-to-vehicle (V2V) cooperative autonomous driving holds great promise for improving safety.<n>Traditional cooperative methods are constrained by rigid collaboration protocols and limited generalization to unseen interactive scenarios.<n>We propose CoLMDriver, the first full-pipeline LLM-based cooperative driving system.
• Score: 28.101283061232472
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vehicle-to-vehicle (V2V) cooperative autonomous driving holds great promise for improving safety by addressing the perception and prediction uncertainties inherent in single-agent systems. However, traditional cooperative methods are constrained by rigid collaboration protocols and limited generalization to unseen interactive scenarios. While LLM-based approaches offer generalized reasoning capabilities, their challenges in spatial planning and unstable inference latency hinder their direct application in cooperative driving. To address these limitations, we propose CoLMDriver, the first full-pipeline LLM-based cooperative driving system, enabling effective language-based negotiation and real-time driving control. CoLMDriver features a parallel driving pipeline with two key components: (i) an LLM-based negotiation module under an actor-critic paradigm, which continuously refines cooperation policies through feedback from previous decisions of all vehicles; and (ii) an intention-guided waypoint generator, which translates negotiation outcomes into executable waypoints. Additionally, we introduce InterDrive, a CARLA-based simulation benchmark comprising 10 challenging interactive driving scenarios for evaluating V2V cooperation. Experimental results demonstrate that CoLMDriver significantly outperforms existing approaches, achieving an 11% higher success rate across diverse highly interactive V2V driving scenarios. Code will be released on https://github.com/cxliu0314/CoLMDriver.

Related papers

• V2V-LLM: Vehicle-to-Vehicle Cooperative Autonomous Driving with Multi-Modal Large Language Models [31.537045261401666]
  Vehicle-to-vehicle (V2V) communication has been proposed, but they have tended to focus on detection and tracking.<n>We propose a novel problem setting that integrates an Large Language Models (LLMs) into cooperative autonomous driving.<n>We also propose our baseline method Vehicle-to-Vehicle Large Language Model (V2V-LLM), which uses an LLM to fuse perception information from multiple connected autonomous vehicles.
  arXiv  Detail & Related papers  (2025-02-14T08:05:41Z)
• TeLL-Drive: Enhancing Autonomous Driving with Teacher LLM-Guided Deep Reinforcement Learning [61.33599727106222]
  TeLL-Drive is a hybrid framework that integrates a Teacher LLM to guide an attention-based Student DRL policy.<n>A self-attention mechanism then fuses these strategies with the DRL agent's exploration, accelerating policy convergence and boosting robustness.
  arXiv  Detail & Related papers  (2025-02-03T14:22:03Z)
• WHALES: A Multi-agent Scheduling Dataset for Enhanced Cooperation in Autonomous Driving [54.365702251769456]
  We present dataset with unprecedented average of 8.4 agents per driving sequence. In addition to providing the largest number of agents and viewpoints among autonomous driving datasets, WHALES records agent behaviors. We conduct experiments on agent scheduling task, where the ego agent selects one of multiple candidate agents to cooperate with.
  arXiv  Detail & Related papers  (2024-11-20T14:12:34Z)
• Towards Interactive and Learnable Cooperative Driving Automation: a Large Language Model-Driven Decision-Making Framework [79.088116316919]
  Connected Autonomous Vehicles (CAVs) have begun to open road testing around the world, but their safety and efficiency performance in complex scenarios is still not satisfactory. This paper proposes CoDrivingLLM, an interactive and learnable LLM-driven cooperative driving framework.
  arXiv  Detail & Related papers  (2024-09-19T14:36:00Z)
• Validation of critical maneuvers based on shared control [0.0]
  This paper presents the validation of shared control strategies for critical maneuvers in automated driving systems. The proposed approach focuses on two critical maneuvers: overtaking in low visibility scenarios and lateral evasive actions. The results demonstrate improved safety and user acceptance, indicating the effectiveness of the shared control strategies.
  arXiv  Detail & Related papers  (2024-04-05T10:38:33Z)
• DriveCoT: Integrating Chain-of-Thought Reasoning with End-to-End Driving [81.04174379726251]
  This paper collects a comprehensive end-to-end driving dataset named DriveCoT. It contains sensor data, control decisions, and chain-of-thought labels to indicate the reasoning process. We propose a baseline model called DriveCoT-Agent, trained on our dataset, to generate chain-of-thought predictions and final decisions.
  arXiv  Detail & Related papers  (2024-03-25T17:59:01Z)
• LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving [87.1164964709168]
  This work employs Large Language Models (LLMs) as a decision-making component for complex autonomous driving scenarios. Extensive experiments demonstrate that our proposed method not only consistently surpasses baseline approaches in single-vehicle tasks, but also helps handle complex driving behaviors even multi-vehicle coordination.
  arXiv  Detail & Related papers  (2023-10-04T17:59:49Z)
• V2X-Lead: LiDAR-based End-to-End Autonomous Driving with Vehicle-to-Everything Communication Integration [4.166623313248682]
  This paper presents a LiDAR-based end-to-end autonomous driving method with Vehicle-to-Everything (V2X) communication integration. The proposed method aims to handle imperfect partial observations by fusing the onboard LiDAR sensor and V2X communication data.
  arXiv  Detail & Related papers  (2023-09-26T20:26:03Z)
• Learning Interactive Driving Policies via Data-driven Simulation [125.97811179463542]
  Data-driven simulators promise high data-efficiency for driving policy learning. Small underlying datasets often lack interesting and challenging edge cases for learning interactive driving. We propose a simulation method that uses in-painted ado vehicles for learning robust driving policies.
  arXiv  Detail & Related papers  (2021-11-23T20:14:02Z)
• Learning Interaction-aware Guidance Policies for Motion Planning in Dense Traffic Scenarios [8.484564880157148]
  This paper presents a novel framework for interaction-aware motion planning in dense traffic scenarios. We propose to learn, via deep Reinforcement Learning (RL), an interaction-aware policy providing global guidance about the cooperativeness of other vehicles. The learned policy can reason and guide the local optimization-based planner with interactive behavior to pro-actively merge in dense traffic while remaining safe in case the other vehicles do not yield.
  arXiv  Detail & Related papers  (2021-07-09T16:43:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.