The EpiBench Platform to Propel AI/ML-based Epidemic Forecasting: A Prototype Demonstration Reaching Human Expert-level Performance

• URL: http://arxiv.org/abs/2102.02842v1
• Date: Thu, 4 Feb 2021 19:21:14 GMT
• Title: The EpiBench Platform to Propel AI/ML-based Epidemic Forecasting: A Prototype Demonstration Reaching Human Expert-level Performance
• Authors: Ajitesh Srivastava, Tianjian Xu, Viktor K. Prasanna
• Abstract summary: During the COVID-19 pandemic, a significant effort has gone into developing ML-driven epidemic forecasting techniques. benchmarks do not exist to claim if a new AI/ML technique is better than the existing ones. We propose EpiBench a platform consisting of community-driven benchmarks for AI/ML applied to epidemic forecasting.
• Score: 9.352573037053448
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: During the COVID-19 pandemic, a significant effort has gone into developing ML-driven epidemic forecasting techniques. However, benchmarks do not exist to claim if a new AI/ML technique is better than the existing ones. The "covid-forecast-hub" is a collection of more than 30 teams, including us, that submit their forecasts weekly to the CDC. It is not possible to declare whether one method is better than the other using those forecasts because each team's submission may correspond to different techniques over the period and involve human interventions as the teams are continuously changing/tuning their approach. Such forecasts may be considered "human-expert" forecasts and do not qualify as AI/ML approaches, although they can be used as an indicator of human expert performance. We are interested in supporting AI/ML research in epidemic forecasting which can lead to scalable forecasting without human intervention. Which modeling technique, learning strategy, and data pre-processing technique work well for epidemic forecasting is still an open problem. To help advance the state-of-the-art AI/ML applied to epidemiology, a benchmark with a collection of performance points is needed and the current "state-of-the-art" techniques need to be identified. We propose EpiBench a platform consisting of community-driven benchmarks for AI/ML applied to epidemic forecasting to standardize the challenge with a uniform evaluation protocol. In this paper, we introduce a prototype of EpiBench which is currently running and accepting submissions for the task of forecasting COVID-19 cases and deaths in the US states and We demonstrate that we can utilize the prototype to develop an ensemble relying on fully automated epidemic forecasts (no human intervention) that reaches human-expert level ensemble currently being used by the CDC.

Related papers

• Forecasting Patient Flows with Pandemic Induced Concept Drift using Explainable Machine Learning [0.0]
  This study investigates how a suite of novel quasi-real-time variables can improve the forecasting models of patient flows. The prevailing COVID-19 Alert Level feature together with Google search terms and pedestrian traffic were effective at producing generalisable forecasts.
  arXiv  Detail & Related papers  (2022-11-01T20:42:26Z)
• Data-Centric Epidemic Forecasting: A Survey [56.99209141838794]
  This survey delves into various data-driven methodological and practical advancements. We enumerate the large number of epidemiological datasets and novel data streams that are relevant to epidemic forecasting. We also discuss experiences and challenges that arise in real-world deployment of these forecasting systems.
  arXiv  Detail & Related papers  (2022-07-19T16:15:11Z)
• Boosting the interpretability of clinical risk scores with intervention predictions [59.22442473992704]
  We propose a joint model of intervention policy and adverse event risk as a means to explicitly communicate the model's assumptions about future interventions. We show how combining typical risk scores, such as the likelihood of mortality, with future intervention probability scores leads to more interpretable clinical predictions.
  arXiv  Detail & Related papers  (2022-07-06T19:49:42Z)
• Predicting infections in the Covid-19 pandemic -- lessons learned [5.981641988736108]
  In this paper, we start from prediction algorithms proposed for XPrize Pandemic Response Challenge and consider several directions that might allow their improvement. We find that augmenting the algorithms with additional information about the culture of the modeled region can improve the performance for short term predictions. The accuracy of medium-term predictions is still very low and a significant amount of future research is needed to make such models a reliable component of a public policy toolbox.
  arXiv  Detail & Related papers  (2021-12-02T20:20:46Z)
• Test-time Collective Prediction [73.74982509510961]
  Multiple parties in machine learning want to jointly make predictions on future test points. Agents wish to benefit from the collective expertise of the full set of agents, but may not be willing to release their data or model parameters. We explore a decentralized mechanism to make collective predictions at test time, leveraging each agent's pre-trained model.
  arXiv  Detail & Related papers  (2021-06-22T18:29:58Z)
• Incorporating Expert Guidance in Epidemic Forecasting [79.91855362871496]
  We propose a new approach leveraging the Seldonian optimization framework from AI safety. We study two types of guidance: smoothness and regional consistency of errors. We show that by its successful incorporation, we are able to not only bound the probability of undesirable behavior to happen, but also to reduce RMSE on test data by up to 17%.
  arXiv  Detail & Related papers  (2020-12-24T06:21:53Z)
• STELAR: Spatio-temporal Tensor Factorization with Latent Epidemiological Regularization [76.57716281104938]
  We develop a tensor method to predict the evolution of epidemic trends for many regions simultaneously. STELAR enables long-term prediction by incorporating latent temporal regularization through a system of discrete-time difference equations. We conduct experiments using both county- and state-level COVID-19 data and show that our model can identify interesting latent patterns of the epidemic.
  arXiv  Detail & Related papers  (2020-12-08T21:21:47Z)
• Forecasting COVID-19 daily cases using phone call data [0.0]
  We propose a simple Multiple Linear Regression model optimised to use call data to forecast the number of daily confirmed cases. Our proposed approach outperforms ARIMA, ETS and a regression model without call data, evaluated by three point forecast error metrics, one prediction interval and two probabilistic forecast accuracy measures.
  arXiv  Detail & Related papers  (2020-10-05T18:07:07Z)
• BiteNet: Bidirectional Temporal Encoder Network to Predict Medical Outcomes [53.163089893876645]
  We propose a novel self-attention mechanism that captures the contextual dependency and temporal relationships within a patient's healthcare journey. An end-to-end bidirectional temporal encoder network (BiteNet) then learns representations of the patient's journeys. We have evaluated the effectiveness of our methods on two supervised prediction and two unsupervised clustering tasks with a real-world EHR dataset.
  arXiv  Detail & Related papers  (2020-09-24T00:42:36Z)
• Learning to Forecast and Forecasting to Learn from the COVID-19 Pandemic [10.796851110372593]
  We propose a heterogeneous infection rate model with human mobility for epidemic modeling. By linearizing the model and using weighted least squares, our model is able to quickly adapt to changing trends. We show that during the earlier part of the epidemic, using travel data increases the predictions.
  arXiv  Detail & Related papers  (2020-04-23T07:25:46Z)
• Machine Learning for Predicting Epileptic Seizures Using EEG Signals: A Review [1.7959899851975951]
  We provide a review of state-of-the-art ML techniques in early prediction of seizures using EEG signals. Patients can be relieved from the adverse consequences of epileptic seizures if it could somehow be predicted in advance. There have been exciting new developments in ML-based algorithms that have the potential to deliver a paradigm shift in the early and accurate prediction of epileptic seizures.
  arXiv  Detail & Related papers  (2020-02-04T06:22:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.