Assessment of Reward Functions in Reinforcement Learning for Multi-Modal Urban Traffic Control under Real-World limitations

• URL: http://arxiv.org/abs/2010.08819v1
• Date: Sat, 17 Oct 2020 16:20:33 GMT
• Title: Assessment of Reward Functions in Reinforcement Learning for Multi-Modal Urban Traffic Control under Real-World limitations
• Authors: Alvaro Cabrejas-Egea, Colm Connaughton
• Abstract summary: This paper robustly evaluates 30 different Reinforcement Learning reward functions for controlling intersections serving pedestrians and vehicles. We use a calibrated model in terms of demand, sensors, green times and other operational constraints of a real intersection in Greater Manchester, UK.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reinforcement Learning is proving a successful tool that can manage urban intersections with a fraction of the effort required to curate traditional traffic controllers. However, literature on the introduction and control of pedestrians to such intersections is scarce. Furthermore, it is unclear what traffic state variables should be used as reward to obtain the best agent performance. This paper robustly evaluates 30 different Reinforcement Learning reward functions for controlling intersections serving pedestrians and vehicles covering the main traffic state variables available via modern vision-based sensors. Some rewards proposed in previous literature solely for vehicular traffic are extended to pedestrians while new ones are introduced. We use a calibrated model in terms of demand, sensors, green times and other operational constraints of a real intersection in Greater Manchester, UK. The assessed rewards can be classified in 5 groups depending on the magnitudes used: queues, waiting time, delay, average speed and throughput in the junction. The performance of different agents, in terms of waiting time, is compared across different demand levels, from normal operation to saturation of traditional adaptive controllers. We find that those rewards maximising the speed of the network obtain the lowest waiting time for vehicles and pedestrians simultaneously, closely followed by queue minimisation, demonstrating better performance than other previously proposed methods.

Related papers

• DenseLight: Efficient Control for Large-scale Traffic Signals with Dense Feedback [109.84667902348498]
  Traffic Signal Control (TSC) aims to reduce the average travel time of vehicles in a road network. Most prior TSC methods leverage deep reinforcement learning to search for a control policy. We propose DenseLight, a novel RL-based TSC method that employs an unbiased reward function to provide dense feedback on policy effectiveness.
  arXiv  Detail & Related papers  (2023-06-13T05:58:57Z)
• SocialLight: Distributed Cooperation Learning towards Network-Wide Traffic Signal Control [7.387226437589183]
  SocialLight is a new multi-agent reinforcement learning method for traffic signal control. It learns cooperative traffic control policies by estimating the individual marginal contribution of agents on their local neighborhood. We benchmark our trained network against state-of-the-art traffic signal control methods on standard benchmarks in two traffic simulators.
  arXiv  Detail & Related papers  (2023-04-20T12:41:25Z)
• Learning energy-efficient driving behaviors by imitating experts [75.12960180185105]
  This paper examines the role of imitation learning in bridging the gap between control strategies and realistic limitations in communication and sensing. We show that imitation learning can succeed in deriving policies that, if adopted by 5% of vehicles, may boost the energy-efficiency of networks with varying traffic conditions by 15% using only local observations.
  arXiv  Detail & Related papers  (2022-06-28T17:08:31Z)
• Pedestrian Stop and Go Forecasting with Hybrid Feature Fusion [87.77727495366702]
  We introduce the new task of pedestrian stop and go forecasting. Considering the lack of suitable existing datasets for it, we release TRANS, a benchmark for explicitly studying the stop and go behaviors of pedestrians in urban traffic. We build it from several existing datasets annotated with pedestrians' walking motions, in order to have various scenarios and behaviors.
  arXiv  Detail & Related papers  (2022-03-04T18:39:31Z)
• Pedestrian Detection: Domain Generalization, CNNs, Transformers and Beyond [82.37430109152383]
  We show that, current pedestrian detectors poorly handle even small domain shifts in cross-dataset evaluation. We attribute the limited generalization to two main factors, the method and the current sources of data. We propose a progressive fine-tuning strategy which improves generalization.
  arXiv  Detail & Related papers  (2022-01-10T06:00:26Z)
• A Deep Value-network Based Approach for Multi-Driver Order Dispatching [55.36656442934531]
  We propose a deep reinforcement learning based solution for order dispatching. We conduct large scale online A/B tests on DiDi's ride-dispatching platform. Results show that CVNet consistently outperforms other recently proposed dispatching methods.
  arXiv  Detail & Related papers  (2021-06-08T16:27:04Z)
• End-to-End Intersection Handling using Multi-Agent Deep Reinforcement Learning [63.56464608571663]
  Navigating through intersections is one of the main challenging tasks for an autonomous vehicle. In this work, we focus on the implementation of a system able to navigate through intersections where only traffic signs are provided. We propose a multi-agent system using a continuous, model-free Deep Reinforcement Learning algorithm used to train a neural network for predicting both the acceleration and the steering angle at each time step.
  arXiv  Detail & Related papers  (2021-04-28T07:54:40Z)
• Assessment of Reward Functions for Reinforcement Learning Traffic Signal Control under Real-World Limitations [0.0]
  This paper compares the performance of agents using different reward functions in a simulation of a junction in Greater Manchester, UK. We find that speed maximisation resulted in the lowest average waiting times across all demand levels, displaying significantly better performance than other rewards previously introduced in the literature.
  arXiv  Detail & Related papers  (2020-08-26T15:47:15Z)
• Optimizing Traffic Lights with Multi-agent Deep Reinforcement Learning and V2X communication [5.40232936994133]
  We consider a system to optimize duration of traffic signals using multi-agent deep reinforcement learning and Vehicle-to-Everything (V2X) communication. This system aims at analyzing independent and shared rewards for multi-agents to control duration of traffic lights.
  arXiv  Detail & Related papers  (2020-02-23T07:43:12Z)
• Decoding pedestrian and automated vehicle interactions using immersive virtual reality and interpretable deep learning [6.982614422666432]
  This study investigates pedestrian crossing behaviour, as an important element of urban dynamics that is expected to be affected by the presence of automated vehicles. Pedestrian wait time behaviour is then analyzed using a data-driven Cox Proportional Hazards (CPH) model. Results show that the presence of automated vehicles on roads, wider lane widths, high density on roads, limited sight distance, and lack of walking habits are the main contributing factors to longer wait times.
  arXiv  Detail & Related papers  (2020-02-18T01:30:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.