Adaptive Double-Booking Strategy for Outpatient Scheduling Using Multi-Objective Reinforcement Learning

• URL: http://arxiv.org/abs/2603.07270v1
• Date: Sat, 07 Mar 2026 15:58:38 GMT
• Title: Adaptive Double-Booking Strategy for Outpatient Scheduling Using Multi-Objective Reinforcement Learning
• Abstract summary: We propose an adaptive outpatient double-booking framework that integrates individualized no-show prediction with reinforcement learning.<n>The proposed framework determines when to single-book, double-book, or reject appointment requests.
• Score: 0.3437656066916039
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Patient no-shows disrupt outpatient clinic operations, reduce productivity, and may delay necessary care. Clinics often adopt overbooking or double-booking to mitigate these effects. However, poorly calibrated policies can increase congestion and waiting times. Most existing methods rely on fixed heuristics and fail to adapt to real-time scheduling conditions or patient-specific no-show risk. To address these limitations, we propose an adaptive outpatient double-booking framework that integrates individualized no-show prediction with multi-objective reinforcement learning. The scheduling problem is formulated as a Markov decision process, and patient-level no-show probabilities estimated by a Multi-Head Attention Soft Random Forest model are incorporated in the reinforcement learning state. We develop a Multi-Policy Proximal Policy Optimization method equipped with a Multi-Policy Co-Evolution Mechanism. Under this mechanism, we propose a novel τ rule based on Kullback-Leibler divergence that enables selective knowledge transfer among behaviorally similar policies, improving convergence and expanding the diversity of trade-offs. In addition, SHapley Additive exPlanations is used to interpret both the predicted no-show risk and the agent's scheduling decisions. The proposed framework determines when to single-book, double-book, or reject appointment requests, providing a dynamic and data-driven alternative to conventional outpatient scheduling policies.

Related papers

• Behavioural Signatures of Risk-Sensitive Decision-Making in Large Language Models [121.70781851874035]
  Large language models (LLMs) are increasingly used in decision support.<n>It is important to understand whether their choices under uncertainty exhibit stable and interpretable behavioural regularities.<n>Here we examine this question using a controlled multi-model framework based on no-limit Texas Hold'em.
  arXiv  Detail & Related papers  (2026-07-11T10:33:39Z)
• Deep Optimal Individualized Treatment Rules for Bivariate Survival Outcomes via Adaptive Prediction-Powered Learning [3.890091933986142]
  This paper addresses the problem of deriving optimal individualized treatment rules to maximize the joint survival probability beyond fixed time points.<n>We propose a novel approach that models treatment rules via policies, coupling marginal accelerated failure time models via link function.<n>To enhance robustness and effectiveness of decision making, we introduce an adaptive prediction-powered method that leverages auxiliary predictions from machine learning models.
  arXiv  Detail & Related papers  (2026-05-28T06:56:52Z)
• PCHC: Enabling Preference Conditioned Humanoid Control via Multi-Objective Reinforcement Learning [50.63196995993855]
  We propose a novel framework leveraging Multi-Objective Reinforcement Learning (MORL) to achieve Preference-Conditioned Humanoid Control (PCHC)<n>Our framework enables a single, preference-conditioned policy to exhibit a wide spectrum of diverse behaviors.
  arXiv  Detail & Related papers  (2026-03-25T07:55:37Z)
• Compositional Planning with Jumpy World Models [70.74595987225908]
  We study agents that compose pre-trained policies as temporally extended actions, enabling solutions to complex tasks that no constituent alone can solve.<n>Motivated by the geometric policy composition framework introduced in arXiv:2206.08736, we address these challenges by learning predictive models of multi-step dynamics.
  arXiv  Detail & Related papers  (2026-02-23T09:22:21Z)
• Multi-Agent Reinforcement Learning for Intraday Operating Rooms Scheduling under Uncertainty [4.5515292789901975]
  Intraday surgical scheduling is a multi-objective decision problem under uncertainty-balancing throughput, urgent and emergency demand, delays, sequence-dependent setups, and overtime.<n>We formulate the problem as a cooperative Markov game and propose a multi-agent reinforcement learning framework in which each operating room is an agent trained with centralized training and decentralized execution.<n>All agents share a policy trained via Proximal Policy Optimization (PPO), which maps rich system states to actions, while a within-epoch sequential assignment protocol constructs joint schedules across ORs.
  arXiv  Detail & Related papers  (2025-12-04T15:47:08Z)
• Power Constrained Nonstationary Bandits with Habituation and Recovery Dynamics [0.9699640804685629]
  This paper develops a Thompson Sampling algorithm tailored to the ROGUE framework.<n>We then introduce a probability clipping procedure to balance personalization and population-level learning.<n>For researchers designing micro-randomized trials, our framework offers practical guidance on balancing personalization with statistical validity.
  arXiv  Detail & Related papers  (2025-11-04T19:46:42Z)
• Offline Multi-agent Reinforcement Learning via Score Decomposition [51.23590397383217]
  offline cooperative multi-agent reinforcement learning (MARL) faces unique challenges due to distributional shifts.<n>This work is the first work to explicitly address the distributional gap between offline and online MARL.
  arXiv  Detail & Related papers  (2025-05-09T11:42:31Z)
• Generalization Bounds of Surrogate Policies for Combinatorial Optimization Problems [53.03951222945921]
  We analyze smoothed (perturbed) policies, adding controlled random perturbations to the direction used by the linear oracle.<n>Our main contribution is a generalization bound that decomposes the excess risk into perturbation bias, statistical estimation error, and optimization error.<n>We illustrate the scope of the results on applications such as vehicle scheduling, highlighting how smoothing enables both tractable training and controlled generalization.
  arXiv  Detail & Related papers  (2024-07-24T12:00:30Z)
• Decentralized Learning Strategies for Estimation Error Minimization with Graph Neural Networks [86.99017195607077]
  We address the challenge of sampling and remote estimation for autoregressive Markovian processes in a wireless network with statistically-identical agents.<n>Our goal is to minimize time-average estimation error and/or age of information with decentralized scalable sampling and transmission policies.
  arXiv  Detail & Related papers  (2024-04-04T06:24:11Z)
• Estimating Optimal Infinite Horizon Dynamic Treatment Regimes via pT-Learning [2.0625936401496237]
  Recent advances in mobile health (mHealth) technology provide an effective way to monitor individuals' health statuses and deliver just-in-time personalized interventions. The practical use of mHealth technology raises unique challenges to existing methodologies on learning an optimal dynamic treatment regime. We propose a Proximal Temporal Learning framework to estimate an optimal regime adaptively adjusted between deterministic and sparse policy models.
  arXiv  Detail & Related papers  (2021-10-20T18:38:22Z)
• State Augmented Constrained Reinforcement Learning: Overcoming the Limitations of Learning with Rewards [88.30521204048551]
  A common formulation of constrained reinforcement learning involves multiple rewards that must individually accumulate to given thresholds. We show a simple example in which the desired optimal policy cannot be induced by any weighted linear combination of rewards. This work addresses this shortcoming by augmenting the state with Lagrange multipliers and reinterpreting primal-dual methods.
  arXiv  Detail & Related papers  (2021-02-23T21:07:35Z)
• Identifying Decision Points for Safe and Interpretable Reinforcement Learning in Hypotension Treatment [24.010934215216142]
  We develop a novel framework to compress continuous trajectories into a few, interpretable decision points. We apply our approach to create recommendations from a cohort of hypotensive patients dataset.
  arXiv  Detail & Related papers  (2021-01-09T07:15:33Z)
• Cautious Reinforcement Learning via Distributional Risk in the Dual Domain [45.17200683056563]
  We study the estimation of risk-sensitive policies in reinforcement learning problems defined by a Markov Decision Process (MDPs) whose state and action spaces are countably finite. We propose a new definition of risk, which we call caution, as a penalty function added to the dual objective of the linear programming (LP) formulation of reinforcement learning.
  arXiv  Detail & Related papers  (2020-02-27T23:18:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.