Autonomous Vehicles Using Multi-Agent Reinforcement Learning for Routing Decisions Can Harm Urban Traffic

• URL: http://arxiv.org/abs/2502.13188v1
• Date: Tue, 18 Feb 2025 13:37:02 GMT
• Title: Autonomous Vehicles Using Multi-Agent Reinforcement Learning for Routing Decisions Can Harm Urban Traffic
• Authors: Anastasia Psarou, Ahmet Onur Akman, Łukasz Gorczyca, Michał Hoffmann, Zoltán György Varga, Grzegorz Jamróz, Rafał Kucharski,
• Abstract summary: Autonomous vehicles (AVs) using Multi-Agent Reinforcement Learning (MARL) for simultaneous route optimization may destabilize traffic environments.<n>Our experiments with standard MARL algorithms reveal that, even in trivial cases, policies often fail to converge to an optimal solution.<n>Future research must prioritize realistic benchmarks, cautious deployment strategies, and tools for monitoring and regulating AV routing behaviors.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Autonomous vehicles (AVs) using Multi-Agent Reinforcement Learning (MARL) for simultaneous route optimization may destabilize traffic environments, with human drivers possibly experiencing longer travel times. We study this interaction by simulating human drivers and AVs. Our experiments with standard MARL algorithms reveal that, even in trivial cases, policies often fail to converge to an optimal solution or require long training periods. The problem is amplified by the fact that we cannot rely entirely on simulated training, as there are no accurate models of human routing behavior. At the same time, real-world training in cities risks destabilizing urban traffic systems, increasing externalities, such as $CO_2$ emissions, and introducing non-stationarity as human drivers adapt unpredictably to AV behaviors. Centralization can improve convergence in some cases, however, it raises privacy concerns for the travelers' destination data. In this position paper, we argue that future research must prioritize realistic benchmarks, cautious deployment strategies, and tools for monitoring and regulating AV routing behaviors to ensure sustainable and equitable urban mobility systems.

Related papers

• URB -- Urban Routing Benchmark for RL-equipped Connected Autonomous Vehicles [0.0]
  Reinforcement learning (RL) can facilitate the development of such collective routing strategies.<n>We present our: Urban Routing Benchmark for RL-equipped Connected Autonomous Vehicles.<n>Our results suggest that, despite the lengthy and costly training, state-of-the-art MARL algorithms rarely outperformed humans.
  arXiv  Detail & Related papers  (2025-05-23T10:54:53Z)
• Origin-Destination Pattern Effects on Large-Scale Mixed Traffic Control via Multi-Agent Reinforcement Learning [7.813738581616868]
  Large-scale mixed traffic control, involving both human-driven and robotic vehicles, remains underexplored.<n>We propose a decentralized multi-agent reinforcement learning framework for managing large-scale mixed traffic networks.<n>We evaluate our approach on a real-world network of 14 intersections in Colorado Springs, Colorado, USA.
  arXiv  Detail & Related papers  (2025-05-19T01:36:05Z)
• CoT-VLM4Tar: Chain-of-Thought Guided Vision-Language Models for Traffic Anomaly Resolution [14.703196966156288]
  CoT-VLM4Tar: (Chain of Thought Visual-Language Model for Traffic Anomaly Resolution)<n>This paper introduces a new chain-of-thought to guide the VLM in analyzing, reasoning, and generating solutions for traffic anomalies with greater reasonable and effective solution.<n>Our results demonstrate the effectiveness of VLM in the resolution of real-time traffic anomalies, providing a proof-of-concept for its integration into autonomous traffic management systems.
  arXiv  Detail & Related papers  (2025-03-03T15:07:25Z)
• Enhancing Safety for Autonomous Agents in Partly Concealed Urban Traffic Environments Through Representation-Based Shielding [2.9685635948300004]
  We propose a novel state representation for Reinforcement Learning (RL) agents centered around the information perceivable by an autonomous agent. Our findings pave the way for more robust and reliable autonomous navigation strategies.
  arXiv  Detail & Related papers  (2024-07-05T08:34:49Z)
• Agent-Agnostic Centralized Training for Decentralized Multi-Agent Cooperative Driving [17.659812774579756]
  We propose an asymmetric actor-critic model that learns decentralized cooperative driving policies for autonomous vehicles. By employing attention neural networks with masking, our approach efficiently manages real-world traffic dynamics and partial observability.
  arXiv  Detail & Related papers  (2024-03-18T16:13:02Z)
• Learning Realistic Traffic Agents in Closed-loop [36.38063449192355]
  Reinforcement learning (RL) can train traffic agents to avoid infractions, but using RL alone results in unhuman-like driving behaviors. We propose Reinforcing Traffic Rules (RTR) to match expert demonstrations under a traffic compliance constraint. Our experiments show that RTR learns more realistic and generalizable traffic simulation policies.
  arXiv  Detail & Related papers  (2023-11-02T16:55:23Z)
• Convergence of Communications, Control, and Machine Learning for Secure and Autonomous Vehicle Navigation [78.60496411542549]
  Connected and autonomous vehicles (CAVs) can reduce human errors in traffic accidents, increase road efficiency, and execute various tasks. Reaping these benefits requires CAVs to autonomously navigate to target destinations. This article proposes solutions using the convergence of communication theory, control theory, and machine learning to enable effective and secure CAV navigation.
  arXiv  Detail & Related papers  (2023-07-05T21:38:36Z)
• Rethinking Closed-loop Training for Autonomous Driving [82.61418945804544]
  We present the first empirical study which analyzes the effects of different training benchmark designs on the success of learning agents. We propose trajectory value learning (TRAVL), an RL-based driving agent that performs planning with multistep look-ahead. Our experiments show that TRAVL can learn much faster and produce safer maneuvers compared to all the baselines.
  arXiv  Detail & Related papers  (2023-06-27T17:58:39Z)
• FastRLAP: A System for Learning High-Speed Driving via Deep RL and Autonomous Practicing [71.76084256567599]
  We present a system that enables an autonomous small-scale RC car to drive aggressively from visual observations using reinforcement learning (RL) Our system, FastRLAP (faster lap), trains autonomously in the real world, without human interventions, and without requiring any simulation or expert demonstrations. The resulting policies exhibit emergent aggressive driving skills, such as timing braking and acceleration around turns and avoiding areas which impede the robot's motion, approaching the performance of a human driver using a similar first-person interface over the course of training.
  arXiv  Detail & Related papers  (2023-04-19T17:33:47Z)
• Generative AI-empowered Simulation for Autonomous Driving in Vehicular Mixed Reality Metaverses [130.15554653948897]
  In vehicular mixed reality (MR) Metaverse, distance between physical and virtual entities can be overcome. Large-scale traffic and driving simulation via realistic data collection and fusion from the physical world is difficult and costly. We propose an autonomous driving architecture, where generative AI is leveraged to synthesize unlimited conditioned traffic and driving data in simulations.
  arXiv  Detail & Related papers  (2023-02-16T16:54:10Z)
• Reinforcement Learning based Cyberattack Model for Adaptive Traffic Signal Controller in Connected Transportation Systems [61.39400591328625]
  In a connected transportation system, adaptive traffic signal controllers (ATSC) utilize real-time vehicle trajectory data received from vehicles to regulate green time. This wirelessly connected ATSC increases cyber-attack surfaces and increases their vulnerability to various cyber-attack modes. One such mode is a'sybil' attack in which an attacker creates fake vehicles in the network. An RL agent is trained to learn an optimal rate of sybil vehicle injection to create congestion for an approach(s)
  arXiv  Detail & Related papers  (2022-10-31T20:12:17Z)
• Learning energy-efficient driving behaviors by imitating experts [75.12960180185105]
  This paper examines the role of imitation learning in bridging the gap between control strategies and realistic limitations in communication and sensing. We show that imitation learning can succeed in deriving policies that, if adopted by 5% of vehicles, may boost the energy-efficiency of networks with varying traffic conditions by 15% using only local observations.
  arXiv  Detail & Related papers  (2022-06-28T17:08:31Z)
• Traffic Management of Autonomous Vehicles using Policy Based Deep Reinforcement Learning and Intelligent Routing [0.26249027950824505]
  We propose a DRL-based signal control system that adjusts traffic signals according to the current congestion situation on intersections. To deal with the congestion on roads behind the intersection, we used re-routing technique to load balance the vehicles on road networks.
  arXiv  Detail & Related papers  (2022-06-28T02:46:20Z)
• Calibration of Human Driving Behavior and Preference Using Naturalistic Traffic Data [5.926030548326619]
  We show how the model can be inverted to estimate driver preferences from naturalistic traffic data. One distinct advantage of our approach is the drastically reduced computational burden.
  arXiv  Detail & Related papers  (2021-05-05T01:20:03Z)
• End-to-End Intersection Handling using Multi-Agent Deep Reinforcement Learning [63.56464608571663]
  Navigating through intersections is one of the main challenging tasks for an autonomous vehicle. In this work, we focus on the implementation of a system able to navigate through intersections where only traffic signs are provided. We propose a multi-agent system using a continuous, model-free Deep Reinforcement Learning algorithm used to train a neural network for predicting both the acceleration and the steering angle at each time step.
  arXiv  Detail & Related papers  (2021-04-28T07:54:40Z)
• Cautious Adaptation For Reinforcement Learning in Safety-Critical Settings [129.80279257258098]
  Reinforcement learning (RL) in real-world safety-critical target settings like urban driving is hazardous. We propose a "safety-critical adaptation" task setting: an agent first trains in non-safety-critical "source" environments. We propose a solution approach, CARL, that builds on the intuition that prior experience in diverse environments equips an agent to estimate risk.
  arXiv  Detail & Related papers  (2020-08-15T01:40:59Z)
• Deep Reinforcement Learning for Human-Like Driving Policies in Collision Avoidance Tasks of Self-Driving Cars [1.160208922584163]
  We introduce a model-free, deep reinforcement learning approach to generate automated human-like driving policies. We study a static obstacle avoidance task on a two-lane highway road in simulation. We demonstrate that our approach leads to human-like driving policies.
  arXiv  Detail & Related papers  (2020-06-07T18:20:33Z)
• Decoding pedestrian and automated vehicle interactions using immersive virtual reality and interpretable deep learning [6.982614422666432]
  This study investigates pedestrian crossing behaviour, as an important element of urban dynamics that is expected to be affected by the presence of automated vehicles. Pedestrian wait time behaviour is then analyzed using a data-driven Cox Proportional Hazards (CPH) model. Results show that the presence of automated vehicles on roads, wider lane widths, high density on roads, limited sight distance, and lack of walking habits are the main contributing factors to longer wait times.
  arXiv  Detail & Related papers  (2020-02-18T01:30:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.