Reward Design for Driver Repositioning Using Multi-Agent Reinforcement Learning

• URL: http://arxiv.org/abs/2002.06723v3
• Date: Sun, 23 Aug 2020 16:48:23 GMT
• Title: Reward Design for Driver Repositioning Using Multi-Agent Reinforcement Learning
• Abstract summary: This paper aims to model the multi-driver repositioning task through a mean field multi-agent reinforcement learning (MARL) approach. Because the direct application of MARL to the multi-driver system under a given reward mechanism will likely yield a suboptimal equilibrium due to the selfishness of drivers, this study proposes a reward design scheme. We apply the bilevel optimization model to two case studies, namely, e-hailing driver repositioning under service charge and multiclass taxi driver repositioning under NYC congestion pricing.
• Score: 3.3123634393237706
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A large portion of passenger requests is reportedly unserviced, partially due to vacant for-hire drivers' cruising behavior during the passenger seeking process. This paper aims to model the multi-driver repositioning task through a mean field multi-agent reinforcement learning (MARL) approach that captures competition among multiple agents. Because the direct application of MARL to the multi-driver system under a given reward mechanism will likely yield a suboptimal equilibrium due to the selfishness of drivers, this study proposes a reward design scheme with which a more desired equilibrium can be reached. To effectively solve the bilevel optimization problem with upper level as the reward design and the lower level as a multi-agent system, a Bayesian optimization (BO) algorithm is adopted to speed up the learning process. We then apply the bilevel optimization model to two case studies, namely, e-hailing driver repositioning under service charge and multiclass taxi driver repositioning under NYC congestion pricing. In the first case study, the model is validated by the agreement between the derived optimal control from BO and that from an analytical solution. With a simple piecewise linear service charge, the objective of the e-hailing platform can be increased by 8.4%. In the second case study, an optimal toll charge of $5.1 is solved using BO, which improves the objective of city planners by 7.9%, compared to that without any toll charge. Under this optimal toll charge, the number of taxis in the NYC central business district is decreased, indicating a better traffic condition, without substantially increasing the crowdedness of the subway system.

Related papers

• Ling and Ring 2.6 Technical Report: Efficient and Instant Agentic Intelligence at Trillion-Parameter Scale [231.1015617648021]
  We present Ling-2.6 and Ring-2.6, a family of models designed to address this challenge at scale.<n>Ling-2.6 is optimized for instant response generation and high capability per output token.<n>Ring-2.6 is tailored for deeper reasoning and more advanced agentic.
  arXiv  Detail & Related papers  (2026-06-13T03:21:49Z)
• Positive-Unlabeled Reinforcement Learning Distillation for On-Premise Small Models [130.8912476550625]
  We propose a positive-unlabeled (PU) reinforcement learning distillation method for on-premise small-model deployment.<n>Our method distills the teacher's preference-optimization capability from black-box generations into a locally trainable student.<n>Experiments demonstrate that our method achieves consistently strong performance under a low-cost setting.
  arXiv  Detail & Related papers  (2026-01-28T15:14:50Z)
• RouteMoA: Dynamic Routing without Pre-Inference Boosts Efficient Mixture-of-Agents [91.0187958746262]
  RouteMoA is an efficient mixture-of-agents framework with dynamic routing.<n>It employs a lightweight scorer to perform initial screening by predicting coarse-grained performance from the query.<n>It refines these scores through lightweight self- and cross-assessment based on existing model outputs, providing posterior correction without additional inference.
  arXiv  Detail & Related papers  (2026-01-26T04:22:22Z)
• SPARK: Synergistic Policy And Reward Co-Evolving Framework [84.22494672256894]
  We introduce the Synergistic Policy And Reward Co-Evolving Framework (SPARK), an efficient, on-policy, and stable method that builds on RLVR.<n>Instead of discarding rollouts and correctness data, SPARK recycles this valuable information to simultaneously train the model itself as a generative reward model.<n>We show that SPARK achieves significant performance gains on multiple LLM and LVLM models and multiple reasoning, reward models, and general benchmarks.
  arXiv  Detail & Related papers  (2025-09-26T17:50:12Z)
• Semi-on-Demand Transit Feeders with Shared Autonomous Vehicles and Reinforcement-Learning-Based Zonal Dispatching Control [2.5407305954218797]
  This paper develops a semi-on-demand transit feeder service using shared autonomous vehicles (SAVs) and zonal dispatching control based on reinforcement learning (RL)<n>After efficient training of the RL model, the semi-on-demand service with dynamic zonal control serves 16% more passengers at 13% higher generalized costs on average compared to traditional fixed-route service.
  arXiv  Detail & Related papers  (2025-09-02T02:14:11Z)
• Bilevel subsidy-enabled mobility hub network design with perturbed utility coalitional choice-based assignment [3.836506019632274]
  We develop a bilevel MH platform design model that treats MHs as control levers.<n>The upper level maximizes revenue or flow by setting subsidies to incentivize last-mile operators.<n>The lower level captures joint traveler-operator decisions with a link-based Perturbed Utility Route Choice assignment.
  arXiv  Detail & Related papers  (2025-08-18T16:34:15Z)
• Process Reinforcement through Implicit Rewards [95.7442934212076]
  Dense process rewards have proven a more effective alternative to the sparse outcome-level rewards in the inference-time scaling of large language models (LLMs) Dense rewards also offer an appealing choice for the reinforcement learning (RL) of LLMs since their fine-grained rewards have the potential to address some inherent issues of outcome rewards. This can be primarily attributed to the challenges of training process reward models (PRMs) online, where collecting high-quality process labels is prohibitively expensive. We propose PRIME, which enables online PRM updates using only policy rollouts and outcome labels through implict process rewards
  arXiv  Detail & Related papers  (2025-02-03T15:43:48Z)
• Online Merging Optimizers for Boosting Rewards and Mitigating Tax in Alignment [47.682736928029996]
  Large Language Models (LLMs) are designed to align with human-centric values while preventing the degradation of abilities acquired through Pre-training and Supervised Fine-tuning (SFT) In this paper, we show that interpolating RLHF and SFT model parameters can adjust the trade-off between human preference and basic capabilities, thereby reducing the alignment tax. It significantly enhances alignment reward while mitigating alignment tax, achieving higher overall performance across 14 benchmarks.
  arXiv  Detail & Related papers  (2024-05-28T07:53:40Z)
• Global Rewards in Multi-Agent Deep Reinforcement Learning for Autonomous Mobility on Demand Systems [3.9271220134003206]
  We study vehicle dispatching in autonomous mobility on demand (AMoD) systems. Recent approaches use multi-agent deep reinforcement learning (MADRL) to realize scalable yet performant algorithms. We propose a novel global-rewards-based MADRL algorithm for vehicle dispatching in AMoD systems.
  arXiv  Detail & Related papers  (2023-12-14T12:47:33Z)
• Coalitional Bargaining via Reinforcement Learning: An Application to Collaborative Vehicle Routing [49.00137468773683]
  Collaborative Vehicle Routing is where delivery companies cooperate by sharing their delivery information and performing delivery requests on behalf of each other. This achieves economies of scale and thus reduces cost, greenhouse gas emissions, and road congestion. But which company should partner with whom, and how much should each company be compensated? Traditional game theoretic solution concepts, such as the Shapley value or nucleolus, are difficult to calculate for the real-world problem of Collaborative Vehicle Routing.
  arXiv  Detail & Related papers  (2023-10-26T15:04:23Z)
• iPLAN: Intent-Aware Planning in Heterogeneous Traffic via Distributed Multi-Agent Reinforcement Learning [57.24340061741223]
  We introduce a distributed multi-agent reinforcement learning (MARL) algorithm that can predict trajectories and intents in dense and heterogeneous traffic scenarios. Our approach for intent-aware planning, iPLAN, allows agents to infer nearby drivers' intents solely from their local observations.
  arXiv  Detail & Related papers  (2023-06-09T20:12:02Z)
• Inducing Equilibria via Incentives: Simultaneous Design-and-Play Finds Global Optima [114.31577038081026]
  We propose an efficient method that tackles the designer's and agents' problems simultaneously in a single loop. Although the designer does not solve the equilibrium problem repeatedly, it can anticipate the overall influence of the incentives on the agents. We prove that the algorithm converges to the global optima at a sublinear rate for a broad class of games.
  arXiv  Detail & Related papers  (2021-10-04T06:53:59Z)
• Value Function is All You Need: A Unified Learning Framework for Ride Hailing Platforms [57.21078336887961]
  Large ride-hailing platforms, such as DiDi, Uber and Lyft, connect tens of thousands of vehicles in a city to millions of ride demands throughout the day. We propose a unified value-based dynamic learning framework (V1D3) for tackling both tasks.
  arXiv  Detail & Related papers  (2021-05-18T19:22:24Z)
• Incentivizing Efficient Equilibria in Traffic Networks with Mixed Autonomy [17.513581783749707]
  Vehicle platooning can potentially reduce traffic congestion by increasing road capacity via vehicle platooning. We consider a network of parallel roads with two modes of transportation: (i) human drivers, who will choose the quickest route available to them, and (ii) a ride hailing service, which provides an array of autonomous vehicle route options, each with different prices, to users. We formalize a model of vehicle flow in mixed autonomy and a model of how autonomous service users make choices between routes with different prices and latencies.
  arXiv  Detail & Related papers  (2021-05-06T03:01:46Z)
• Balancing Fairness and Efficiency in Traffic Routing via Interpolated Traffic Assignment [29.556405472628402]
  Interpolated Traffic Assignment Problem (I-TAP) is a convex program that interpolates between a fairness-promoting and an efficiency-promoting traffic-assignment objective. We present a numerical comparison between I-TAP and a state-of-the-art algorithm on a range of transportation networks.
  arXiv  Detail & Related papers  (2021-03-31T20:32:52Z)
• Optimizing Mixed Autonomy Traffic Flow With Decentralized Autonomous Vehicles and Multi-Agent RL [63.52264764099532]
  We study the ability of autonomous vehicles to improve the throughput of a bottleneck using a fully decentralized control scheme in a mixed autonomy setting. We apply multi-agent reinforcement algorithms to this problem and demonstrate that significant improvements in bottleneck throughput, from 20% at a 5% penetration rate to 33% at a 40% penetration rate, can be achieved.
  arXiv  Detail & Related papers  (2020-10-30T22:06:05Z)
• A Distributed Model-Free Ride-Sharing Approach for Joint Matching, Pricing, and Dispatching using Deep Reinforcement Learning [32.0512015286512]
  We present a dynamic, demand aware, and pricing-based vehicle-passenger matching and route planning framework. Our framework is validated using the New York City Taxi dataset. Experimental results show the effectiveness of our approach in real-time and large scale settings.
  arXiv  Detail & Related papers  (2020-10-05T03:13:47Z)
• Real-time and Large-scale Fleet Allocation of Autonomous Taxis: A Case Study in New York Manhattan Island [14.501650948647324]
  Traditional models fail to efficiently allocate the available fleet to deal with the imbalance of supply (autonomous taxis) and demand (trips) We employ a Constrained Multi-agent Markov Decision Processes (CMMDP) to model fleet allocation decisions. We also leverage a Column Generation algorithm to guarantee the efficiency and optimality in a large scale.
  arXiv  Detail & Related papers  (2020-09-06T16:00:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.