Optimizing Resource-Constrained Non-Pharmaceutical Interventions for Multi-Cluster Outbreak Control Using Hierarchical Reinforcement Learning

• URL: http://arxiv.org/abs/2603.19397v1
• Date: Thu, 19 Mar 2026 18:38:05 GMT
• Title: Optimizing Resource-Constrained Non-Pharmaceutical Interventions for Multi-Cluster Outbreak Control Using Hierarchical Reinforcement Learning
• Abstract summary: Non-pharmaceutical interventions, such as diagnostic testing and quarantine, are crucial for controlling infectious disease outbreaks.<n>In real-world public health settings, resources must be allocated across multiple outbreak clusters that emerge asynchronously, vary in size and risk, and compete for a shared resource budget.<n>We formulate this problem as a constrained restless multi-armed bandit and propose a hierarchical reinforcement learning framework.
• Score: 12.292485152613935
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Non-pharmaceutical interventions (NPIs), such as diagnostic testing and quarantine, are crucial for controlling infectious disease outbreaks but are often constrained by limited resources, particularly in early outbreak stages. In real-world public health settings, resources must be allocated across multiple outbreak clusters that emerge asynchronously, vary in size and risk, and compete for a shared resource budget. Here, a cluster corresponds to a group of close contacts generated by a single infected index case. Thus, decisions must be made under uncertainty and heterogeneous demands, while respecting operational constraints. We formulate this problem as a constrained restless multi-armed bandit and propose a hierarchical reinforcement learning framework. A global controller learns a continuous action cost multiplier that adjusts global resource demand, while a generalized local policy estimates the marginal value of allocating resources to individuals within each cluster. We evaluate the proposed framework in a realistic agent-based simulator of SARS-CoV-2 with dynamically arriving clusters. Across a wide range of system scales and testing budgets, our method consistently outperforms RMAB-inspired and heuristic baselines, improving outbreak control effectiveness by 20%-30%. Experiments on up to 40 concurrently active clusters further demonstrate that the hierarchical framework is highly scalable and enables faster decision-making than the RMAB-inspired method.

Related papers

• Sampling-Based Coordination-Informed Multi-Objective Multi-Robot Reinforcement Learning [16.418881902763502]
  Multi-robot systems must simultaneously optimize competing objectives while maintaining coordinated behavior.<n>This work introduces the Coordination-Informed Multi-Objective Reinforcement Learning (CIMORL) framework.<n>We present the base CIMORL method alongside two sampling-based variants, CIMORL-TS (Tree Search) and CIMORL-MPPI (MPPI)
  arXiv  Detail & Related papers  (2026-06-29T20:31:03Z)
• Boosted Distributional Reinforcement Learning: Analysis and Healthcare Applications [0.8348593305367524]
  We propose a Boosted Distributional Reinforcement Learning algorithm to optimize decisions in complex domains like robotics and healthcare.<n>We apply our algorithm to manage hypertension in a large subset of the US adult population by categorizing individuals into cardiovascular disease risk groups.
  arXiv  Detail & Related papers  (2026-04-06T01:00:08Z)
• Adversarially Robust Multitask Adaptive Control [6.576173998482649]
  We study adversarially robust multitask adaptive linear quadratic control.<n>We propose a clustered multitask approach that integrates clustering and system identification with resilient aggregation to mitigate corrupted model updates.
  arXiv  Detail & Related papers  (2025-11-07T17:25:21Z)
• Neural Index Policies for Restless Multi-Action Bandits with Heterogeneous Budgets [2.9059410824803655]
  We introduce a Neural Index Policy (NIP) for multi-action RMABs with heterogeneous budget constraints.<n>NIP unifies index prediction and constrained optimization in a single end-to-end differentiable framework.<n> Empirically, NIP achieves near-optimal performance within 5% of the occupancy oracle-measure policy.
  arXiv  Detail & Related papers  (2025-10-24T23:08:36Z)
• Offline Multi-agent Reinforcement Learning via Score Decomposition [51.23590397383217]
  offline cooperative multi-agent reinforcement learning (MARL) faces unique challenges due to distributional shifts.<n>This work is the first work to explicitly address the distributional gap between offline and online MARL.
  arXiv  Detail & Related papers  (2025-05-09T11:42:31Z)
• Decentralized Reinforcement Learning for Multi-Agent Multi-Resource Allocation via Dynamic Cluster Agreements [13.586104024751586]
  This paper addresses the challenge of allocating heterogeneous resources among multiple agents in a decentralized manner.<n>Our proposed method, LGTC-IPPO, builds upon Independent Proximal Policy Optimization (IPPO) by integrating dynamic cluster consensus.<n> Experimental results demonstrate that LGTC-IPPO achieves more stable rewards, better coordination, and robust performance even as the number of agents or resource types increases.
  arXiv  Detail & Related papers  (2025-03-04T09:23:48Z)
• Deep Reinforcement Learning for Efficient and Fair Allocation of Health Care Resources [47.57108369791273]
  Scarcity of health care resources could result in the unavoidable consequence of rationing. There is no universally accepted standard for health care resource allocation protocols. We propose a transformer-based deep Q-network to integrate the disease progression of individual patients and the interaction effects among patients.
  arXiv  Detail & Related papers  (2023-09-15T17:28:06Z)
• Safe Multi-agent Learning via Trapping Regions [89.24858306636816]
  We apply the concept of trapping regions, known from qualitative theory of dynamical systems, to create safety sets in the joint strategy space for decentralized learning. We propose a binary partitioning algorithm for verification that candidate sets form trapping regions in systems with known learning dynamics, and a sampling algorithm for scenarios where learning dynamics are not known.
  arXiv  Detail & Related papers  (2023-02-27T14:47:52Z)
• Optimal Clustering with Bandit Feedback [57.672609011609886]
  This paper considers the problem of online clustering with bandit feedback. It includes a novel stopping rule for sequential testing that circumvents the need to solve any NP-hard weighted clustering problem as its subroutines. We show through extensive simulations on synthetic and real-world datasets that BOC's performance matches the lower boundally, and significantly outperforms a non-adaptive baseline algorithm.
  arXiv  Detail & Related papers  (2022-02-09T06:05:05Z)
• Reinforcement Learning with Heterogeneous Data: Estimation and Inference [84.72174994749305]
  We introduce the K-Heterogeneous Markov Decision Process (K-Hetero MDP) to address sequential decision problems with population heterogeneity. We propose the Auto-Clustered Policy Evaluation (ACPE) for estimating the value of a given policy, and the Auto-Clustered Policy Iteration (ACPI) for estimating the optimal policy in a given policy class. We present simulations to support our theoretical findings, and we conduct an empirical study on the standard MIMIC-III dataset.
  arXiv  Detail & Related papers  (2022-01-31T20:58:47Z)
• Convergence Rates of Average-Reward Multi-agent Reinforcement Learning via Randomized Linear Programming [41.30044824711509]
  We focus on the case that the global reward is a sum of local rewards, the joint policy factorizes into agents' marginals, and full state observability. We develop multi-agent extensions, whereby agents solve their local saddle point problems and then perform local weighted averaging. We establish that the sample complexity to obtain near-globally optimal solutions matches tight dependencies on the cardinality of the state and action spaces.
  arXiv  Detail & Related papers  (2021-10-22T03:48:41Z)
• Hybrid Dynamic Contrast and Probability Distillation for Unsupervised Person Re-Id [109.1730454118532]
  Unsupervised person re-identification (Re-Id) has attracted increasing attention due to its practical application in the read-world video surveillance system. We present the hybrid dynamic cluster contrast and probability distillation algorithm. It formulates the unsupervised Re-Id problem into an unified local-to-global dynamic contrastive learning and self-supervised probability distillation framework.
  arXiv  Detail & Related papers  (2021-09-29T02:56:45Z)
• Active Screening for Recurrent Diseases: A Reinforcement Learning Approach [29.78172882606022]
  We propose a novel reinforcement learning (RL) approach based on Deep Q-Networks (DQN) We evaluate our RL algorithm on several real-world networks.
  arXiv  Detail & Related papers  (2021-01-07T21:07:35Z)
• Coordinated Online Learning for Multi-Agent Systems with Coupled Constraints and Perturbed Utility Observations [91.02019381927236]
  We introduce a novel method to steer the agents toward a stable population state, fulfilling the given resource constraints. The proposed method is a decentralized resource pricing method based on the resource loads resulting from the augmentation of the game's Lagrangian.
  arXiv  Detail & Related papers  (2020-10-21T10:11:17Z)
• F2A2: Flexible Fully-decentralized Approximate Actor-critic for Cooperative Multi-agent Reinforcement Learning [110.35516334788687]
  Decentralized multi-agent reinforcement learning algorithms are sometimes unpractical in complicated applications. We propose a flexible fully decentralized actor-critic MARL framework, which can handle large-scale general cooperative multi-agent setting. Our framework can achieve scalability and stability for large-scale environment and reduce information transmission.
  arXiv  Detail & Related papers  (2020-04-17T14:56:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.