A Microscopic Pandemic Simulator for Pandemic Prediction Using Scalable Million-Agent Reinforcement Learning

• URL: http://arxiv.org/abs/2108.06589v1
• Date: Sat, 14 Aug 2021 17:07:25 GMT
• Title: A Microscopic Pandemic Simulator for Pandemic Prediction Using Scalable Million-Agent Reinforcement Learning
• Authors: Zhenggang Tang, Kai Yan, Liting Sun, Wei Zhan, Changliu Liu
• Abstract summary: This paper proposes a deep-reinforcement-learning-powered microscopic model named Microscopic Pandemic Simulator (MPS) By replacing rule-based agents with rational agents whose behaviors are driven to maximize rewards, the MPS provides a better approximation of real world dynamics. This paper first calibrates the MPS against real-world data in Allegheny, US, then demonstratively evaluates two government strategies: information disclosure and quarantine.
• Score: 7.653466578233261
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Microscopic epidemic models are powerful tools for government policy makers to predict and simulate epidemic outbreaks, which can capture the impact of individual behaviors on the macroscopic phenomenon. However, existing models only consider simple rule-based individual behaviors, limiting their applicability. This paper proposes a deep-reinforcement-learning-powered microscopic model named Microscopic Pandemic Simulator (MPS). By replacing rule-based agents with rational agents whose behaviors are driven to maximize rewards, the MPS provides a better approximation of real world dynamics. To efficiently simulate with massive amounts of agents in MPS, we propose Scalable Million-Agent DQN (SMADQN). The MPS allows us to efficiently evaluate the impact of different government strategies. This paper first calibrates the MPS against real-world data in Allegheny, US, then demonstratively evaluates two government strategies: information disclosure and quarantine. The results validate the effectiveness of the proposed method. As a broad impact, this paper provides novel insights for the application of DRL in large scale agent-based networks such as economic and social networks.

Related papers

• Self-Healing Machine Learning: A Framework for Autonomous Adaptation in Real-World Environments [50.310636905746975]
  Real-world machine learning systems often encounter model performance degradation due to distributional shifts in the underlying data generating process. Existing approaches to addressing shifts, such as concept drift adaptation, are limited by their reason-agnostic nature. We propose self-healing machine learning (SHML) to overcome these limitations.
  arXiv  Detail & Related papers  (2024-10-31T20:05:51Z)
• Modeling and Optimization of Epidemiological Control Policies Through Reinforcement Learning [0.0]
  The impact of a pandemic can be minimized by enforcing certain restrictions on a community. While minimizing infection and death rates, these restrictions can also lead to economic crises. We train a reinforcement learning agent to enforce the optimal restriction on a pandemic simulation based on a reward function.
  arXiv  Detail & Related papers  (2024-01-25T22:39:39Z)
• Self-Supervised Neuron Segmentation with Multi-Agent Reinforcement Learning [53.00683059396803]
  Mask image model (MIM) has been widely used due to its simplicity and effectiveness in recovering original information from masked images. We propose a decision-based MIM that utilizes reinforcement learning (RL) to automatically search for optimal image masking ratio and masking strategy. Our approach has a significant advantage over alternative self-supervised methods on the task of neuron segmentation.
  arXiv  Detail & Related papers  (2023-10-06T10:40:46Z)
• Agent-Based Model: Simulating a Virus Expansion Based on the Acceptance of Containment Measures [65.62256987706128]
  Compartmental epidemiological models categorize individuals based on their disease status. We propose an ABM architecture that combines an adapted SEIRD model with a decision-making model for citizens. We illustrate the designed model by examining the progression of SARS-CoV-2 infections in A Coruna, Spain.
  arXiv  Detail & Related papers  (2023-07-28T08:01:05Z)
• Epidemic Control on a Large-Scale-Agent-Based Epidemiology Model using Deep Deterministic Policy Gradient [0.7244731714427565]
  lockdowns, rapid vaccination programs, school closures, and economic stimulus can have positive or unintended negative consequences. Current research to model and determine an optimal intervention automatically through round-tripping is limited by the simulation objectives, scale (a few thousand individuals), model types that are not suited for intervention studies, and the number of intervention strategies they can explore (discrete vs continuous). We address these challenges using a Deep Deterministic Policy Gradient (DDPG) based policy optimization framework on a large-scale (100,000 individual) epidemiological agent-based simulation.
  arXiv  Detail & Related papers  (2023-04-10T09:26:07Z)
• Differentiable Agent-based Epidemiology [71.81552021144589]
  We introduce GradABM: a scalable, differentiable design for agent-based modeling that is amenable to gradient-based learning with automatic differentiation. GradABM can quickly simulate million-size populations in few seconds on commodity hardware, integrate with deep neural networks and ingest heterogeneous data sources.
  arXiv  Detail & Related papers  (2022-07-20T07:32:02Z)
• ERMAS: Becoming Robust to Reward Function Sim-to-Real Gaps in Multi-Agent Simulations [110.72725220033983]
  Epsilon-Robust Multi-Agent Simulation (ERMAS) is a framework for learning AI policies that are robust to such multiagent sim-to-real gaps. ERMAS learns tax policies that are robust to changes in agent risk aversion, improving social welfare by up to 15% in complextemporal simulations. In particular, ERMAS learns tax policies that are robust to changes in agent risk aversion, improving social welfare by up to 15% in complextemporal simulations.
  arXiv  Detail & Related papers  (2021-06-10T04:32:20Z)
• Pyfectious: An individual-level simulator to discover optimal containment polices for epidemic diseases [16.28189705178286]
  We introduce a simulator capable of modeling a population structure and controlling the disease's propagation at an individualistic level. To showcase potential applications, the simulator parameters are set based on the formal statistics of the COVID-19 pandemic. The simulator is used as the environment of a reinforcement learning problem to find the optimal policies to control the pandemic.
  arXiv  Detail & Related papers  (2021-03-24T10:54:46Z)
• Reinforcement Learning for Optimization of COVID-19 Mitigation policies [29.4529156655747]
  The year 2020 has seen the COVID-19 virus lead to one of the worst global pandemics in history. Governments around the world are faced with the challenge of protecting public health, while keeping the economy running to the greatest extent possible. Epidemiological models provide insight into the spread of these types of diseases and predict the effects of possible intervention policies.
  arXiv  Detail & Related papers  (2020-10-20T18:40:15Z)
• Data-driven Simulation and Optimization for Covid-19 Exit Strategies [16.31545249131776]
  The rapid spread of the Coronavirus SARS-2 is a major challenge that led almost all governments worldwide to take drastic measures to respond to the tragedy. We have built a pandemic simulation and forecasting toolkit that combines a deep learning estimation of the epidemiological parameters of the disease.
  arXiv  Detail & Related papers  (2020-06-12T11:18:25Z)
• A Deep Q-learning/genetic Algorithms Based Novel Methodology For Optimizing Covid-19 Pandemic Government Actions [63.669642197519934]
  We use the SEIR epidemiological model to represent the evolution of the virus COVID-19 over time in the population. The sequences of actions (confinement, self-isolation, two-meter distance or not taking restrictions) are evaluated according to a reward system. We prove that our methodology is a valid tool to discover actions governments can take to reduce the negative effects of a pandemic in both senses.
  arXiv  Detail & Related papers  (2020-05-15T17:17:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.