IntersectionZoo: Eco-driving for Benchmarking Multi-Agent Contextual Reinforcement Learning

• URL: http://arxiv.org/abs/2410.15221v1
• Date: Sat, 19 Oct 2024 21:34:24 GMT
• Title: IntersectionZoo: Eco-driving for Benchmarking Multi-Agent Contextual Reinforcement Learning
• Authors: Vindula Jayawardana, Baptiste Freydt, Ao Qu, Cameron Hickert, Zhongxia Yan, Cathy Wu,
• Abstract summary: We propose IntersectionZoo, a comprehensive benchmark suite for multi-agent reinforcement learning. By grounding IntersectionZoo in a real-world application, we naturally capture real-world problem characteristics. IntersectionZoo is built on data-informed simulations of 16,334 signalized intersections from 10 major US cities.
• Score: 4.80862277413422
• License:
• Abstract: Despite the popularity of multi-agent reinforcement learning (RL) in simulated and two-player applications, its success in messy real-world applications has been limited. A key challenge lies in its generalizability across problem variations, a common necessity for many real-world problems. Contextual reinforcement learning (CRL) formalizes learning policies that generalize across problem variations. However, the lack of standardized benchmarks for multi-agent CRL has hindered progress in the field. Such benchmarks are desired to be based on real-world applications to naturally capture the many open challenges of real-world problems that affect generalization. To bridge this gap, we propose IntersectionZoo, a comprehensive benchmark suite for multi-agent CRL through the real-world application of cooperative eco-driving in urban road networks. The task of cooperative eco-driving is to control a fleet of vehicles to reduce fleet-level vehicular emissions. By grounding IntersectionZoo in a real-world application, we naturally capture real-world problem characteristics, such as partial observability and multiple competing objectives. IntersectionZoo is built on data-informed simulations of 16,334 signalized intersections derived from 10 major US cities, modeled in an open-source industry-grade microscopic traffic simulator. By modeling factors affecting vehicular exhaust emissions (e.g., temperature, road conditions, travel demand), IntersectionZoo provides one million data-driven traffic scenarios. Using these traffic scenarios, we benchmark popular multi-agent RL and human-like driving algorithms and demonstrate that the popular multi-agent RL algorithms struggle to generalize in CRL settings.

Related papers

• Multi-Agent Environments for Vehicle Routing Problems [1.0179489519625304]
  We propose a library composed of multi-agent environments that simulates classic vehicle routing problems. The library, built on PyTorch, provides a flexible modular architecture design that allows easy customization and incorporation of new routing problems.
  arXiv  Detail & Related papers  (2024-11-21T18:46:23Z)
• Generalizing Cooperative Eco-driving via Multi-residual Task Learning [6.864745785996583]
  Multi-residual Task Learning (MRTL) is a generic learning framework based on multi-task learning. MRTL decomposes control into nominal components that are effectively solved by conventional control methods and residual terms. We employ MRTL for fleet-level emission reduction in mixed traffic using autonomous vehicles as a means of system control.
  arXiv  Detail & Related papers  (2024-03-07T05:25:34Z)
• Safe Model-Based Multi-Agent Mean-Field Reinforcement Learning [48.667697255912614]
  Mean-field reinforcement learning addresses the policy of a representative agent interacting with the infinite population of identical agents. We propose Safe-M$3$-UCRL, the first model-based mean-field reinforcement learning algorithm that attains safe policies even in the case of unknown transitions. Our algorithm effectively meets the demand in critical areas while ensuring service accessibility in regions with low demand.
  arXiv  Detail & Related papers  (2023-06-29T15:57:07Z)
• Driver Dojo: A Benchmark for Generalizable Reinforcement Learning for Autonomous Driving [1.496194593196997]
  We propose a benchmark for generalizable reinforcement learning for autonomous driving. Our application-oriented benchmark enables a better understanding of the impact of design decisions. Our benchmark aims to encourage researchers to propose solutions that are able to successfully generalize across scenarios.
  arXiv  Detail & Related papers  (2022-07-23T06:29:43Z)
• Graph Convolution-Based Deep Reinforcement Learning for Multi-Agent Decision-Making in Mixed Traffic Environments [12.34509371288594]
  This research proposes a framework to enable different Graph Reinforcement Learning (GRL) methods for decision-making. Several GRL approaches are summarized and implemented in the proposed framework. Results are analyzed in multiple perspectives and dimensions to compare the characteristic of different GRL approaches in intelligent transportation scenarios.
  arXiv  Detail & Related papers  (2022-01-30T10:09:43Z)
• Automated Reinforcement Learning (AutoRL): A Survey and Open Problems [92.73407630874841]
  Automated Reinforcement Learning (AutoRL) involves not only standard applications of AutoML but also includes additional challenges unique to RL. We provide a common taxonomy, discuss each area in detail and pose open problems which would be of interest to researchers going forward.
  arXiv  Detail & Related papers  (2022-01-11T12:41:43Z)
• End-to-End Intersection Handling using Multi-Agent Deep Reinforcement Learning [63.56464608571663]
  Navigating through intersections is one of the main challenging tasks for an autonomous vehicle. In this work, we focus on the implementation of a system able to navigate through intersections where only traffic signs are provided. We propose a multi-agent system using a continuous, model-free Deep Reinforcement Learning algorithm used to train a neural network for predicting both the acceleration and the steering angle at each time step.
  arXiv  Detail & Related papers  (2021-04-28T07:54:40Z)
• Independent Reinforcement Learning for Weakly Cooperative Multiagent Traffic Control Problem [22.733542222812158]
  We use independent reinforcement learning (IRL) to solve a complex traffic cooperative control problem in this study. To this, we model the traffic control problem as a partially observable weak cooperative traffic model (PO-WCTM) to optimize the overall traffic situation of a group of intersections. Experimental results show that CIL-DDQN outperforms other methods in almost all performance indicators of the traffic control problem.
  arXiv  Detail & Related papers  (2021-04-22T07:55:46Z)
• Multi-intersection Traffic Optimisation: A Benchmark Dataset and a Strong Baseline [85.9210953301628]
  Control of traffic signals is fundamental and critical to alleviate traffic congestion in urban areas. Because of the high complexity of modelling the problem, experimental settings of current works are often inconsistent. We propose a novel and strong baseline model based on deep reinforcement learning with the encoder-decoder structure.
  arXiv  Detail & Related papers  (2021-01-24T03:55:39Z)
• MetaVIM: Meta Variationally Intrinsic Motivated Reinforcement Learning for Decentralized Traffic Signal Control [54.162449208797334]
  Traffic signal control aims to coordinate traffic signals across intersections to improve the traffic efficiency of a district or a city. Deep reinforcement learning (RL) has been applied to traffic signal control recently and demonstrated promising performance where each traffic signal is regarded as an agent. We propose a novel Meta Variationally Intrinsic Motivated (MetaVIM) RL method to learn the decentralized policy for each intersection that considers neighbor information in a latent way.
  arXiv  Detail & Related papers  (2021-01-04T03:06:08Z)
• Reinforcement Learning Based Vehicle-cell Association Algorithm for Highly Mobile Millimeter Wave Communication [53.47785498477648]
  This paper investigates the problem of vehicle-cell association in millimeter wave (mmWave) communication networks. We first formulate the user state (VU) problem as a discrete non-vehicle association optimization problem. The proposed solution achieves up to 15% gains in terms sum of user complexity and 20% reduction in VUE compared to several baseline designs.
  arXiv  Detail & Related papers  (2020-01-22T08:51:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.