Reward Engineering for Spatial Epidemic Simulations: A Reinforcement Learning Platform for Individual Behavioral Learning

• URL: http://arxiv.org/abs/2511.18000v1
• Date: Sat, 22 Nov 2025 10:02:37 GMT
• Title: Reward Engineering for Spatial Epidemic Simulations: A Reinforcement Learning Platform for Individual Behavioral Learning
• Abstract summary: We present ContagionRL, a Gymnasium-compatible reinforcement learning platform specifically designed for systematic reward engineering.<n>Our platform enables rigorous evaluation of how reward function design affects learned survival strategies across diverse epidemic scenarios.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: We present ContagionRL, a Gymnasium-compatible reinforcement learning platform specifically designed for systematic reward engineering in spatial epidemic simulations. Unlike traditional agent-based models that rely on fixed behavioral rules, our platform enables rigorous evaluation of how reward function design affects learned survival strategies across diverse epidemic scenarios. ContagionRL integrates a spatial SIRS+D epidemiological model with configurable environmental parameters, allowing researchers to stress-test reward functions under varying conditions including limited observability, different movement patterns, and heterogeneous population dynamics. We evaluate five distinct reward designs, ranging from sparse survival bonuses to a novel potential field approach, across multiple RL algorithms (PPO, SAC, A2C). Through systematic ablation studies, we identify that directional guidance and explicit adherence incentives are critical components for robust policy learning. Our comprehensive evaluation across varying infection rates, grid sizes, visibility constraints, and movement patterns reveals that reward function choice dramatically impacts agent behavior and survival outcomes. Agents trained with our potential field reward consistently achieve superior performance, learning maximal adherence to non-pharmaceutical interventions while developing sophisticated spatial avoidance strategies. The platform's modular design enables systematic exploration of reward-behavior relationships, addressing a knowledge gap in models of this type where reward engineering has received limited attention. ContagionRL is an effective platform for studying adaptive behavioral responses in epidemic contexts and highlight the importance of reward design, information structure, and environmental predictability in learning.

Related papers

• Evolutionary Recurrent Decision Model in Developing Adaptive and Maladaptive Behaviors [0.0]
  This study introduces the evolutionarily recurrent decision model (ERDM)<n>ERDM is a computational reinforcement learning framework designed to examine how evolutionary mismatch, bounded rationality, and satisficing contribute to adaptive and maladaptive behavior.
  arXiv  Detail & Related papers  (2026-08-25T00:45:05Z)
• ICPRL: Acquiring Physical Intuition from Interactive Control [38.098959182766144]
  We introduce ICPRL (In-Context Physical Reinforcement Learning), a framework that empowers VLMs to acquire physical intuition and adapt their policies in-context.<n>Our approach trains a vision-grounded policy model via multi-turn Group Relative Policy Optimization (GRPO) over diverse multi-episode interaction histories.<n>This enables the agent to adapt strategies by conditioning on past trial-and-error sequences, without requiring any weight updates.
  arXiv  Detail & Related papers  (2026-03-01T15:17:18Z)
• Causal Reinforcement Learning based Agent-Patient Interaction with Clinical Domain Knowledge [3.2434118923825483]
  Reinforcement Learning (RL) faces significant challenges in adaptive healthcare interventions, such as dementia care, where data is scarce.<n>We present a novel framework called Causal structure-aware Reinforcement Learning (CRL) that explicitly integrates causal discovery and reasoning into policy optimization.
  arXiv  Detail & Related papers  (2025-11-17T22:38:03Z)
• Integrating Genomics into Multimodal EHR Foundation Models [56.31910745104141]
  This paper introduces an innovative EHR foundation model that integrates Polygenic Risk Scores (PRS) as a foundational data modality.<n>The framework aims to learn complex relationships between clinical data and genetic predispositions.<n>This approach is pivotal for unlocking new insights into disease prediction, proactive health management, risk stratification, and personalized treatment strategies.
  arXiv  Detail & Related papers  (2025-10-24T15:56:40Z)
• Social World Model-Augmented Mechanism Design Policy Learning [58.739456918502704]
  We introduce SWM-AP (Social World Model-Augmented Mechanism Design Policy Learning), which learns a social world model hierarchically to enhance mechanism design.<n>We show that SWM-AP outperforms established model-based and model-free RL baselines in cumulative rewards and sample efficiency.
  arXiv  Detail & Related papers  (2025-10-22T06:01:21Z)
• Deep RL Needs Deep Behavior Analysis: Exploring Implicit Planning by Model-Free Agents in Open-Ended Environments [1.6576957162725725]
  We apply tools from neuroscience and ethology to study DRL agents in a novel, complex, partially observable environment.<n>We use this environment as a platform for applying joint behavioral and neural analysis to agents.<n>Contrary to common assumptions, we find that model-free RNN-based DRL agents can exhibit structured, planning-like behavior.
  arXiv  Detail & Related papers  (2025-06-08T03:43:48Z)
• Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Models [79.2162092822111]
  We systematically evaluate reinforcement learning (RL) and control-based methods on a suite of navigation tasks.<n>We employ a latent dynamics model using the Joint Embedding Predictive Architecture (JEPA) and employ it for planning.<n>Our results show that model-free RL benefits most from large amounts of high-quality data, whereas model-based planning generalizes better to unseen layouts.
  arXiv  Detail & Related papers  (2025-02-20T18:39:41Z)
• Towards Empowerment Gain through Causal Structure Learning in Model-Based RL [35.933469787075]
  We propose a novel framework, Empowerment through Causal Learning (ECL), to improve learning efficiency and controllability.<n>ECL operates by first training a causal dynamics model of the environment based on collected data.<n>We then maximize empowerment under the causal structure for exploration, simultaneously using data gathered through exploration to update causal dynamics model to be more controllable.
  arXiv  Detail & Related papers  (2025-02-14T10:59:09Z)
• A Foundational Brain Dynamics Model via Stochastic Optimal Control [15.8358479596609]
  We introduce a foundational model for brain dynamics that utilizes optimal control (SOC) and amortized inference.<n>Our method features a continuous-discrete state space model (SSM) that can robustly handle the intricate and noisy nature of fMRI signals.<n>Our model attains state-of-the-art results across a variety of downstream tasks, including demographic prediction, trait analysis, disease diagnosis, and prognosis.
  arXiv  Detail & Related papers  (2025-02-07T12:57:26Z)
• Efficient Adaptation in Mixed-Motive Environments via Hierarchical Opponent Modeling and Planning [51.52387511006586]
  We propose Hierarchical Opponent modeling and Planning (HOP), a novel multi-agent decision-making algorithm. HOP is hierarchically composed of two modules: an opponent modeling module that infers others' goals and learns corresponding goal-conditioned policies. HOP exhibits superior few-shot adaptation capabilities when interacting with various unseen agents, and excels in self-play scenarios.
  arXiv  Detail & Related papers  (2024-06-12T08:48:06Z)
• Amortized Active Causal Induction with Deep Reinforcement Learning [20.74277346825603]
  Causal Amortized Active Structure Learning (CAASL) is an active intervention design policy that can select interventions that are adaptive, real-time and that does not require access to the likelihood. This policy is trained with reinforcement learning on a simulator of the design environment, and a reward function that measures how close the true causal graph is to a causal graph posterior inferred from the gathered data.
  arXiv  Detail & Related papers  (2024-05-26T23:14:37Z)
• Spatio-temporal Value Semantics-based Abstraction for Dense Deep Reinforcement Learning [1.4542411354617986]
  Intelligent Cyber-Physical Systems (ICPS) represent a specialized form of Cyber-Physical System (CPS) CNNs and Deep Reinforcement Learning (DRL) undertake multifaceted tasks encompassing perception, decision-making, and control. DRL confronts challenges in terms of efficiency, generalization capabilities, and data scarcity during decision-making process. We propose an innovative abstract modeling approach grounded in spatial-temporal value semantics.
  arXiv  Detail & Related papers  (2024-05-24T02:21:10Z)
• Entropy-Regularized Token-Level Policy Optimization for Language Agent Reinforcement [67.1393112206885]
  Large Language Models (LLMs) have shown promise as intelligent agents in interactive decision-making tasks. We introduce Entropy-Regularized Token-level Policy Optimization (ETPO), an entropy-augmented RL method tailored for optimizing LLMs at the token level. We assess the effectiveness of ETPO within a simulated environment that models data science code generation as a series of multi-step interactive tasks.
  arXiv  Detail & Related papers  (2024-02-09T07:45:26Z)
• Continual Visual Reinforcement Learning with A Life-Long World Model [55.05017177980985]
  We present a new continual learning approach for visual dynamics modeling.<n>We first introduce the life-long world model, which learns task-specific latent dynamics.<n>Then, we address the value estimation challenge for previous tasks with the exploratory-conservative behavior learning approach.
  arXiv  Detail & Related papers  (2023-03-12T05:08:03Z)
• Ecological Reinforcement Learning [76.9893572776141]
  We study the kinds of environment properties that can make learning under such conditions easier. understanding how properties of the environment impact the performance of reinforcement learning agents can help us to structure our tasks in ways that make learning tractable.
  arXiv  Detail & Related papers  (2020-06-22T17:55:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.