Trust-MARL: Trust-Based Multi-Agent Reinforcement Learning Framework for Cooperative On-Ramp Merging Control in Heterogeneous Traffic Flow

• URL: http://arxiv.org/abs/2506.12600v1
• Date: Sat, 14 Jun 2025 18:35:10 GMT
• Title: Trust-MARL: Trust-Based Multi-Agent Reinforcement Learning Framework for Cooperative On-Ramp Merging Control in Heterogeneous Traffic Flow
• Authors: Jie Pan, Tianyi Wang, Christian Claudel, Jing Shi,
• Abstract summary: This study proposes a trust-based multi-agent reinforcement learning (Trust-MARL) framework to address the challenge of cooperative on-ramp merging.<n>Trust-MARL enhances global traffic efficiency by leveraging inter-agent trust to improve bottleneck throughput and mitigate traffic shockwave.<n>An extensive set of ablation studies and comparative experiments validates the effectiveness of the proposed Trust-MARL approach.
• Score: 8.452082837356162
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Intelligent transportation systems require connected and automated vehicles (CAVs) to conduct safe and efficient cooperation with human-driven vehicles (HVs) in complex real-world traffic environments. However, the inherent unpredictability of human behaviour, especially at bottlenecks such as highway on-ramp merging areas, often disrupts traffic flow and compromises system performance. To address the challenge of cooperative on-ramp merging in heterogeneous traffic environments, this study proposes a trust-based multi-agent reinforcement learning (Trust-MARL) framework. At the macro level, Trust-MARL enhances global traffic efficiency by leveraging inter-agent trust to improve bottleneck throughput and mitigate traffic shockwave through emergent group-level coordination. At the micro level, a dynamic trust mechanism is designed to enable CAVs to adjust their cooperative strategies in response to real-time behaviors and historical interactions with both HVs and other CAVs. Furthermore, a trust-triggered game-theoretic decision-making module is integrated to guide each CAV in adapting its cooperation factor and executing context-aware lane-changing decisions under safety, comfort, and efficiency constraints. An extensive set of ablation studies and comparative experiments validates the effectiveness of the proposed Trust-MARL approach, demonstrating significant improvements in safety, efficiency, comfort, and adaptability across varying CAV penetration rates and traffic densities.

Related papers

• HCOMC: A Hierarchical Cooperative On-Ramp Merging Control Framework in Mixed Traffic Environment on Two-Lane Highways [6.838130893718755]
  A cooperative control strategy based on connected and automated vehicles (CAVs) is a fundamental solution to this problem.<n>While CAVs are not fully widespread, it is necessary to propose a hierarchical cooperative on-ramp merging control (HCOMC) framework for heterogeneous traffic flow.<n>This paper proposes a HCOMC framework, consisting of a hierarchical cooperative planning model based on the modified virtual vehicle model, a discretionary lane-changing model based on game theory, and a multi-dominated optimization model.
  arXiv  Detail & Related papers  (2025-07-15T18:01:29Z)
• TGLD: A Trust-Aware Game-Theoretic Lane-Changing Decision Framework for Automated Vehicles in Heterogeneous Traffic [6.838130893718755]
  This study proposes a trust-aware game-theoretic lane-changing decision framework (TGLD)<n>First, we formulate a multi-vehicle coalition game, incorporating fully cooperative interactions among AVs and partially cooperative behaviors from HVs informed by real-time trust evaluations.<n>Second, we develop an online trust evaluation method to dynamically estimate HVs' trust levels during lane-changing interactions, guiding AVs to select context-appropriate cooperative maneuvers.<n>Third, social compatibility objectives are considered by minimizing disruption to surrounding vehicles and enhancing the predictability of AV behaviors.
  arXiv  Detail & Related papers  (2025-07-14T09:01:00Z)
• Confidence-Regulated Generative Diffusion Models for Reliable AI Agent Migration in Vehicular Metaverses [55.70043755630583]
  vehicular AI agents are endowed with environment perception, decision-making, and action execution capabilities.<n>We propose a reliable vehicular AI agent migration framework, achieving reliable dynamic migration and efficient resource scheduling.<n>We develop a Confidence-regulated Generative Diffusion Model (CGDM) to efficiently generate AI agent migration decisions.
  arXiv  Detail & Related papers  (2025-05-19T05:04:48Z)
• TeLL-Drive: Enhancing Autonomous Driving with Teacher LLM-Guided Deep Reinforcement Learning [61.33599727106222]
  TeLL-Drive is a hybrid framework that integrates a Teacher LLM to guide an attention-based Student DRL policy.<n>A self-attention mechanism then fuses these strategies with the DRL agent's exploration, accelerating policy convergence and boosting robustness.
  arXiv  Detail & Related papers  (2025-02-03T14:22:03Z)
• Automated Vehicles at Unsignalized Intersections: Safety and Efficiency Implications of Mixed-Human-Automated Traffic [6.9492069439607995]
  The integration of automated vehicles (AVs) into transportation systems presents an unprecedented opportunity to enhance road safety and efficiency.<n>This study aims to bridge the gap by examining behavioral differences and adaptations of AVs and human-driven vehicles (HVs) at unsignalized intersections.<n>The findings reveal a paradox in mixed traffic flow: while AVs maintain larger safety margins, their conservative behavior can lead to unexpected situations for human drivers.
  arXiv  Detail & Related papers  (2024-10-16T13:19:32Z)
• Towards Interactive and Learnable Cooperative Driving Automation: a Large Language Model-Driven Decision-Making Framework [79.088116316919]
  Connected Autonomous Vehicles (CAVs) have begun to open road testing around the world, but their safety and efficiency performance in complex scenarios is still not satisfactory. This paper proposes CoDrivingLLM, an interactive and learnable LLM-driven cooperative driving framework.
  arXiv  Detail & Related papers  (2024-09-19T14:36:00Z)
• A Conflicts-free, Speed-lossless KAN-based Reinforcement Learning Decision System for Interactive Driving in Roundabouts [17.434924472015812]
  This paper introduces a learning-based algorithm tailored to foster safe and efficient driving behaviors in roundabouts. The proposed algorithm employs a deep Q-learning network to learn safe and efficient driving strategies in complex multi-vehicle roundabouts. The results show that our proposed system consistently achieves safe and efficient driving whilst maintaining a stable training process.
  arXiv  Detail & Related papers  (2024-08-15T16:10:25Z)
• Communication-Aware Reinforcement Learning for Cooperative Adaptive Cruise Control [15.31488551912888]
  Reinforcement Learning (RL) has proven effective in optimizing complex decision-making processes in CACC. MARL has shown remarkable potential by enabling coordinated actions among multiple CAVs. We propose Communication-Aware Reinforcement Learning (CA-RL) to address these challenges.
  arXiv  Detail & Related papers  (2024-07-12T03:28:24Z)
• Convergence of Communications, Control, and Machine Learning for Secure and Autonomous Vehicle Navigation [78.60496411542549]
  Connected and autonomous vehicles (CAVs) can reduce human errors in traffic accidents, increase road efficiency, and execute various tasks. Reaping these benefits requires CAVs to autonomously navigate to target destinations. This article proposes solutions using the convergence of communication theory, control theory, and machine learning to enable effective and secure CAV navigation.
  arXiv  Detail & Related papers  (2023-07-05T21:38:36Z)
• Spatial-Temporal-Aware Safe Multi-Agent Reinforcement Learning of Connected Autonomous Vehicles in Challenging Scenarios [10.37986799561165]
  Communication technologies enable coordination among connected and autonomous vehicles (CAVs) We propose a framework of constrained multi-agent reinforcement learning (MARL) with a parallel safety shield for CAVs. Results show that our proposed methodology significantly increases system safety and efficiency in challenging scenarios.
  arXiv  Detail & Related papers  (2022-10-05T14:39:07Z)
• Towards Robust On-Ramp Merging via Augmented Multimodal Reinforcement Learning [9.48157144651867]
  We present a novel approach for Robust on-ramp merge of CAVs via Augmented and Multi-modal Reinforcement Learning. Specifically, we formulate the on-ramp merging problem as a Markov decision process (MDP) by taking driving safety, comfort driving behavior, and traffic efficiency into account. To provide reliable merging maneuvers, we simultaneously leverage BSM and surveillance images for multi-modal observation.
  arXiv  Detail & Related papers  (2022-07-21T16:34:57Z)
• Transferable Deep Reinforcement Learning Framework for Autonomous Vehicles with Joint Radar-Data Communications [69.24726496448713]
  We propose an intelligent optimization framework based on the Markov Decision Process (MDP) to help the AV make optimal decisions. We then develop an effective learning algorithm leveraging recent advances of deep reinforcement learning techniques to find the optimal policy for the AV. We show that the proposed transferable deep reinforcement learning framework reduces the obstacle miss detection probability by the AV up to 67% compared to other conventional deep reinforcement learning approaches.
  arXiv  Detail & Related papers  (2021-05-28T08:45:37Z)
• Federated Learning on the Road: Autonomous Controller Design for Connected and Autonomous Vehicles [109.71532364079711]
  A new federated learning (FL) framework is proposed for designing the autonomous controller of connected and autonomous vehicles (CAVs) A novel dynamic federated proximal (DFP) algorithm is proposed that accounts for the mobility of CAVs, the wireless fading channels, and the unbalanced and nonindependent and identically distributed data across CAVs. A rigorous convergence analysis is performed for the proposed algorithm to identify how fast the CAVs converge to using the optimal controller.
  arXiv  Detail & Related papers  (2021-02-05T19:57:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.