Automated Vehicles Should be Connected with Natural Language

• URL: http://arxiv.org/abs/2507.01059v1
• Date: Sun, 29 Jun 2025 16:41:19 GMT
• Title: Automated Vehicles Should be Connected with Natural Language
• Authors: Xiangbo Gao, Keshu Wu, Hao Zhang, Kexin Tian, Yang Zhou, Zhengzhong Tu,
• Abstract summary: Multi-agent collaborative driving promises improvements in traffic safety and efficiency through collective perception and decision making.<n>Existing communication media suffer limitations in bandwidth efficiency, information completeness, and agent interoperability.<n>We argue that addressing these challenges requires a transition from purely perception-oriented data exchanges to explicit intent and reasoning communication using natural language.
• Score: 10.579888130257185
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multi-agent collaborative driving promises improvements in traffic safety and efficiency through collective perception and decision making. However, existing communication media -- including raw sensor data, neural network features, and perception results -- suffer limitations in bandwidth efficiency, information completeness, and agent interoperability. Moreover, traditional approaches have largely ignored decision-level fusion, neglecting critical dimensions of collaborative driving. In this paper we argue that addressing these challenges requires a transition from purely perception-oriented data exchanges to explicit intent and reasoning communication using natural language. Natural language balances semantic density and communication bandwidth, adapts flexibly to real-time conditions, and bridges heterogeneous agent platforms. By enabling the direct communication of intentions, rationales, and decisions, it transforms collaborative driving from reactive perception-data sharing into proactive coordination, advancing safety, efficiency, and transparency in intelligent transportation systems.

Related papers

• Semantic Communication-Enhanced Split Federated Learning for Vehicular Networks: Architecture, Challenges, and Case Study [50.345531105285524]
  Vehicular edge intelligence (VEI) is vital for future intelligent transportation systems.<n>Traditional centralized learning in dynamic vehicular networks faces significant communication overhead and privacy risks.<n>This paper presents a semantic communication-enhanced split federated learning (SC-USFL) framework.
  arXiv  Detail & Related papers  (2026-03-05T08:36:49Z)
• UNCAP: Uncertainty-Guided Planning Using Natural Language Communication for Cooperative Autonomous Vehicles [79.10221881250759]
  Uncertainty-Guided Natural Language Cooperative Autonomous Planning (UNCAP) is a vision-language model-based planning approach.<n>It enables CAVs to communicate via lightweight natural language messages while explicitly accounting for perception uncertainty in decision-making.<n> Experiments across diverse driving scenarios show a 63% reduction in communication bandwidth with a 31% increase in driving safety score, a 61% reduction in decision uncertainty, and a four-fold increase in collision distance margin.
  arXiv  Detail & Related papers  (2025-10-14T21:09:09Z)
• Task-Oriented Semantic Communication in Large Multimodal Models-based Vehicle Networks [55.32199894495722]
  We investigate an LMM-based vehicle AI assistant using a Large Language and Vision Assistant (LLaVA)<n>To reduce computational demands and shorten response time, we optimize LLaVA's image slicing to selectively focus on areas of utmost interest to users.<n>We construct a Visual Question Answering (VQA) dataset for traffic scenarios to evaluate effectiveness.
  arXiv  Detail & Related papers  (2025-05-05T07:18:47Z)
• Is Intermediate Fusion All You Need for UAV-based Collaborative Perception? [1.8689461238197957]
  We propose a novel communication-efficient collaborative perception framework based on late-intermediate fusion, dubbed LIF.<n>We leverage vision-guided positional embedding (VPE) and box-based virtual augmented feature (BoBEV) to effectively integrate complementary information from various agents.<n> Experimental results demonstrate that our LIF achieves superior performance with minimal communication bandwidth, proving its effectiveness and practicality.
  arXiv  Detail & Related papers  (2025-04-30T16:22:14Z)
• SPformer: A Transformer Based DRL Decision Making Method for Connected Automated Vehicles [9.840325772591024]
  We propose a CAV decision-making architecture based on transformer and reinforcement learning algorithms. A learnable policy token is used as the learning medium of the multi-vehicle joint policy. Our model can make good use of all the state information of vehicles in traffic scenario.
  arXiv  Detail & Related papers  (2024-09-23T15:16:35Z)
• Semantic Communication for Cooperative Perception using HARQ [51.148203799109304]
  We leverage an importance map to distill critical semantic information, introducing a cooperative perception semantic communication framework. To counter the challenges posed by time-varying multipath fading, our approach incorporates the use of frequency-division multiplexing (OFDM) along with channel estimation and equalization strategies. We introduce a novel semantic error detection method that is integrated with our semantic communication framework in the spirit of hybrid automatic repeated request (HARQ)
  arXiv  Detail & Related papers  (2024-08-29T08:53:26Z)
• An Agile Adaptation Method for Multi-mode Vehicle Communication Networks [9.632025797373158]
  Decision process and reinforcement learning are applied to establish an agile adaptation mechanism. Q-learning is used to train the agile adaptation reinforcement learning model and output the trained model.
  arXiv  Detail & Related papers  (2024-07-18T13:04:34Z)
• Collaborative Perception for Connected and Autonomous Driving: Challenges, Possible Solutions and Opportunities [10.020039700138692]
  Collaborative perception with connected and autonomous vehicles (CAVs) shows a promising solution to overcoming these limitations.<n>In this article, we first identify the challenges of collaborative perception, such as data sharing asynchrony, data volume, and pose errors.<n>We propose a scheme to deal with communication efficiency and latency problems, which is a channel-aware collaborative perception framework.
  arXiv  Detail & Related papers  (2024-01-03T05:33:14Z)
• Selective Communication for Cooperative Perception in End-to-End Autonomous Driving [8.680676599607123]
  We propose a novel selective communication algorithm for cooperative perception. Our algorithm is shown to produce higher success rates than a random selection approach on previously studied safety-critical driving scenario simulations.
  arXiv  Detail & Related papers  (2023-05-26T18:13:17Z)
• Interruption-Aware Cooperative Perception for V2X Communication-Aided Autonomous Driving [49.42873226593071]
  We propose V2X communication INterruption-aware COoperative Perception (V2X-INCOP) for V2X communication-aided autonomous driving. We use historical cooperation information to recover missing information due to the interruptions and alleviate the impact of the interruption issue. Experiments on three public cooperative perception datasets demonstrate that the proposed method is effective in alleviating the impacts of communication interruption on cooperative perception.
  arXiv  Detail & Related papers  (2023-04-24T04:59:13Z)
• Cognitive Semantic Communication Systems Driven by Knowledge Graph: Principle, Implementation, and Performance Evaluation [74.38561925376996]
  Two cognitive semantic communication frameworks are proposed for the single-user and multiple-user communication scenarios. An effective semantic correction algorithm is proposed by mining the inference rule from the knowledge graph. For the multi-user cognitive semantic communication system, a message recovery algorithm is proposed to distinguish messages of different users.
  arXiv  Detail & Related papers  (2023-03-15T12:01:43Z)
• Over-communicate no more: Situated RL agents learn concise communication protocols [78.28898217947467]
  It is unclear how to design artificial agents that can learn to effectively and efficiently communicate with each other. Much research on communication emergence uses reinforcement learning (RL) We explore situated communication in a multi-step task, where the acting agent has to forgo an environmental action to communicate. We find that while all tested pressures can disincentivise over-communication, situated communication does it most effectively and, unlike the cost on effort, does not negatively impact emergence.
  arXiv  Detail & Related papers  (2022-11-02T21:08:14Z)
• Learning to Communicate and Correct Pose Errors [75.03747122616605]
  We study the setting proposed in V2VNet, where nearby self-driving vehicles jointly perform object detection and motion forecasting in a cooperative manner. We propose a novel neural reasoning framework that learns to communicate, to estimate potential errors, and to reach a consensus about those errors.
  arXiv  Detail & Related papers  (2020-11-10T18:19:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.