Multi-Agent Reinforcement Learning from Delayed Marketplace Feedback for Objective-Weight Adaptation in Three-Sided Dispatch

• URL: http://arxiv.org/abs/2606.13604v1
• Date: Thu, 11 Jun 2026 17:21:20 GMT
• Title: Multi-Agent Reinforcement Learning from Delayed Marketplace Feedback for Objective-Weight Adaptation in Three-Sided Dispatch
• Abstract summary: We present a deployed reinforcement learning system at DoorDash that adapts dispatch objective weights in a large-scale food-delivery marketplace using delayed signals.<n>This interface enables offline policy learning under noisy, delayed, and coupled feedback.<n>Results illustrate how world feedback from a live economic and logistics system can be used to safely adapt decision policies online.
• Score: 8.850889334502645
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Dispatch in three-sided marketplaces provides a natural setting for reinforcement learning from world feedback: decisions are evaluated by delayed operational outcomes such as delivery speed, courier utilization, and merchant congestion. We present a deployed reinforcement learning system at DoorDash that adapts dispatch objective weights in a large-scale food-delivery marketplace using delayed signals. Rather than replacing the combinatorial assignment optimizer, a store-level policy learned from logged marketplace data selects a discrete multiplier that shifts the dispatch optimizer's tradeoff between delivery quality and batching efficiency. This interface enables offline policy learning under noisy, delayed, and coupled feedback while preserving production feasibility constraints and operational safeguards. We train a shared value function using centralized offline data and decentralized store-level execution, with Double Q-learning targets and a conservative regularizer to reduce out-of-distribution value overestimation. In a production switchback experiment, the offline-trained policy increases batching and reduces courier-side time costs without degrading customer-facing delivery quality. Results illustrate how world feedback from a live economic and logistics system can be used to safely adapt decision policies online.

Related papers

• From Prediction to Incrementality: Causal Optimization for Large-Scale Targeting and Recommendation [1.8192758370546824]
  We present a decision-centric framework that instead optimize causal effects under global constraints.<n>We evaluate it with offline simulations on a public bandit dataset, targeted architectural ablations, and an online A/B test on LinkedIn Feed marketing traffic.<n>The end-to-end treatment policy delivered a statistically significant $+7.20%$ lift in the primary long-term-value metric.
  arXiv  Detail & Related papers  (2026-08-10T19:54:08Z)
• MA-HEAD-Net: Adaptive Rule-Guided Multi-Agent DRL for AoI Minimization in UAV-Assisted Emergency Networks [71.01350912967922]
  In post-disaster scenarios, unmanned aerial vehicles (UAVs) are critical for establishing emergency communication networks.<n>This paper investigates age of information (AoI) minimization for UAV-assisted emergency communications with heterogeneous emergency services.<n>We develop MA-HEAD-Net, an adaptive rule-guided multi-agent deep reinforcement learning framework.
  arXiv  Detail & Related papers  (2026-08-02T10:05:50Z)
• Self-evolving LLM agents with in-distribution Optimization [60.05867547965365]
  Large Language Models (LLMs) have recently emerged as powerful controllers for interactive agents in complex environments.<n>We propose Q-Evolve, a self-evolving framework for LLM agents that unifies automatic process-reward labeling and policy learning.<n>We evaluate our method on AlfWorld, WebShop, and ScienceWorld, showing Q-Evolve outperforms strong baselines in sample efficiency, robustness, and overall task performance.
  arXiv  Detail & Related papers  (2026-06-05T15:09:52Z)
• When Valid Signals Fail: Regime Boundaries Between LLM Features and RL Trading Policies [0.30458514384586394]
  Large language models (LLMs) generate continuous numerical features that improve reinforcement learning (RL) trading agents.<n>We build a modular pipeline where a frozen LLM serves as a stateless feature extractor, transforming unstructured daily news and filings into a fixed-dimensional vector consumed by a downstream PPO agent.<n>Our findings highlight a gap between feature-level validity and policy-level robustness that parallels known challenges in transfer learning under distribution shift.
  arXiv  Detail & Related papers  (2026-04-13T04:53:06Z)
• OpComm: A Reinforcement Learning Framework for Adaptive Buffer Control in Warehouse Volume Forecasting [28.715625330574966]
  OpComm is a forecasting and decision-support framework that combines supervised learning with reinforcement learning-based buffer control.<n>Across 400+ stations, OpComm reduced weighted Absolute Percentage Error (WAPE) by 21.65% compared to manual forecasts.
  arXiv  Detail & Related papers  (2025-12-17T17:21:19Z)
• Learning to Route LLMs from Bandit Feedback: One Policy, Many Trade-offs [69.2486294522259]
  BaRP is a Bandit Routing-feedback with Preferences approach that trains under the same partial-feedback restriction as deployment.<n> Framed as a contextual bandit over prompt features and a user preference vector, our method simulates an online feedback setting during training and adapts its routing decisions to each new prompt.
  arXiv  Detail & Related papers  (2025-10-08T18:24:59Z)
• Joint Matching and Pricing for Crowd-shipping with In-store Customers [2.7950888004779064]
  This paper examines the use of in-store customers as delivery couriers in a centralized crowd-shipping system.<n>We propose a Markov Decision Process (MDP) model that captures key uncertainties, including the arrival of orders and crowd-shippers.<n>We show that the integrated NeurADP + DDQN policy achieves notable improvements in delivery cost efficiency.
  arXiv  Detail & Related papers  (2025-07-02T14:27:32Z)
• Self-Regulation and Requesting Interventions [63.5863047447313]
  We propose an offline framework that trains a "helper" policy to request interventions.<n>We score optimal intervention timing with PRMs and train the helper model on these labeled trajectories.<n>This offline approach significantly reduces costly intervention calls during training.
  arXiv  Detail & Related papers  (2025-02-07T00:06:17Z)
• Real-Time Integrated Dispatching and Idle Fleet Steering with Deep Reinforcement Learning for A Meal Delivery Platform [0.0]
  This study sets out to solve the real-time order dispatching and idle courier steering problems for a meal delivery platform.<n>We propose a reinforcement learning (RL)-based strategic dual-control framework.<n>We find the delivery efficiency and fairness of workload distribution among couriers have been improved.
  arXiv  Detail & Related papers  (2025-01-10T09:15:40Z)
• FIRE: A Failure-Adaptive Reinforcement Learning Framework for Edge Computing Migrations [54.34189781923818]
  FIRE is a framework that adapts to rare events by training a RL policy in an edge computing digital twin environment.<n>We propose ImRE, an importance sampling-based Q-learning algorithm, which samples rare events proportionally to their impact on the value function.<n>We show that FIRE reduces costs compared to vanilla RL and the greedy baseline in the event of failures.
  arXiv  Detail & Related papers  (2022-09-28T19:49:39Z)
• Dynamic Attention-based Communication-Efficient Federated Learning [85.18941440826309]
  Federated learning (FL) offers a solution to train a global machine learning model. FL suffers performance degradation when client data distribution is non-IID. We propose a new adaptive training algorithm $textttAdaFL$ to combat this degradation.
  arXiv  Detail & Related papers  (2021-08-12T14:18:05Z)
• A Deep Value-network Based Approach for Multi-Driver Order Dispatching [55.36656442934531]
  We propose a deep reinforcement learning based solution for order dispatching. We conduct large scale online A/B tests on DiDi's ride-dispatching platform. Results show that CVNet consistently outperforms other recently proposed dispatching methods.
  arXiv  Detail & Related papers  (2021-06-08T16:27:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.