Empirical Quantitative Analysis of COVID-19 Forecasting Models

• URL: http://arxiv.org/abs/2110.00174v1
• Date: Fri, 1 Oct 2021 02:31:56 GMT
• Title: Empirical Quantitative Analysis of COVID-19 Forecasting Models
• Authors: Yun Zhao, Yuqing Wang, Junfeng Liu, Haotian Xia, Zhenni Xu, Qinghang Hong, Zhiyang Zhou, Linda Petzold
• Abstract summary: COVID-19 has been a public health emergency of international concern since early 2020. No forecasting model appears to be the best for all scenarios. Model selection is the dominant factor in determining the predictive performance.
• Score: 7.744521846416669
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: COVID-19 has been a public health emergency of international concern since early 2020. Reliable forecasting is critical to diminish the impact of this disease. To date, a large number of different forecasting models have been proposed, mainly including statistical models, compartmental models, and deep learning models. However, due to various uncertain factors across different regions such as economics and government policy, no forecasting model appears to be the best for all scenarios. In this paper, we perform quantitative analysis of COVID-19 forecasting of confirmed cases and deaths across different regions in the United States with different forecasting horizons, and evaluate the relative impacts of the following three dimensions on the predictive performance (improvement and variation) through different evaluation metrics: model selection, hyperparameter tuning, and the length of time series required for training. We find that if a dimension brings about higher performance gains, if not well-tuned, it may also lead to harsher performance penalties. Furthermore, model selection is the dominant factor in determining the predictive performance. It is responsible for both the largest improvement and the largest variation in performance in all prediction tasks across different regions. While practitioners may perform more complicated time series analysis in practice, they should be able to achieve reasonable results if they have adequate insight into key decisions like model selection.

Related papers

• Rating Multi-Modal Time-Series Forecasting Models (MM-TSFM) for Robustness Through a Causal Lens [10.103561529332184]
  We focus on multi-modal time-series forecasting, where imprecision due to noisy or incorrect data can lead to erroneous predictions. We introduce a rating methodology to assess the robustness of Multi-Modal Time-Series Forecasting Models.
  arXiv  Detail & Related papers  (2024-06-12T17:39:16Z)
• Predictive Churn with the Set of Good Models [64.05949860750235]
  We study the effect of conflicting predictions over the set of near-optimal machine learning models. We present theoretical results on the expected churn between models within the Rashomon set. We show how our approach can be used to better anticipate, reduce, and avoid churn in consumer-facing applications.
  arXiv  Detail & Related papers  (2024-02-12T16:15:25Z)
• A comparison of short-term probabilistic forecasts for the incidence of COVID-19 using mechanistic and statistical time series models [15.031837435365532]
  Short-term forecasts of infectious disease spread are a critical component in risk evaluation and public health decision making. We compare short-term probabilistic forecasts of popular mechanistic models with forecasts of statistical time series models. We find that, on average, probabilistic forecasts from statistical time series models are overall at least as accurate as forecasts from mechanistic models.
  arXiv  Detail & Related papers  (2023-05-01T16:37:43Z)
• Back2Future: Leveraging Backfill Dynamics for Improving Real-time Predictions in Future [73.03458424369657]
  In real-time forecasting in public health, data collection is a non-trivial and demanding task. 'Backfill' phenomenon and its effect on model performance has been barely studied in the prior literature. We formulate a novel problem and neural framework Back2Future that aims to refine a given model's predictions in real-time.
  arXiv  Detail & Related papers  (2021-06-08T14:48:20Z)
• When in Doubt: Neural Non-Parametric Uncertainty Quantification for Epidemic Forecasting [70.54920804222031]
  Most existing forecasting models disregard uncertainty quantification, resulting in mis-calibrated predictions. Recent works in deep neural models for uncertainty-aware time-series forecasting also have several limitations. We model the forecasting task as a probabilistic generative process and propose a functional neural process model called EPIFNP.
  arXiv  Detail & Related papers  (2021-06-07T18:31:47Z)
• Characterizing Fairness Over the Set of Good Models Under Selective Labels [69.64662540443162]
  We develop a framework for characterizing predictive fairness properties over the set of models that deliver similar overall performance. We provide tractable algorithms to compute the range of attainable group-level predictive disparities. We extend our framework to address the empirically relevant challenge of selectively labelled data.
  arXiv  Detail & Related papers  (2021-01-02T02:11:37Z)
• A Worrying Analysis of Probabilistic Time-series Models for Sales Forecasting [10.690379201437015]
  Probabilistic time-series models become popular in the forecasting field as they help to make optimal decisions under uncertainty. We analyze the performance of three prominent probabilistic time-series models for sales forecasting.
  arXiv  Detail & Related papers  (2020-11-21T03:31:23Z)
• Forecasting COVID-19 daily cases using phone call data [0.0]
  We propose a simple Multiple Linear Regression model optimised to use call data to forecast the number of daily confirmed cases. Our proposed approach outperforms ARIMA, ETS and a regression model without call data, evaluated by three point forecast error metrics, one prediction interval and two probabilistic forecast accuracy measures.
  arXiv  Detail & Related papers  (2020-10-05T18:07:07Z)
• Counterfactual Predictions under Runtime Confounding [74.90756694584839]
  We study the counterfactual prediction task in the setting where all relevant factors are captured in the historical data. We propose a doubly-robust procedure for learning counterfactual prediction models in this setting.
  arXiv  Detail & Related papers  (2020-06-30T15:49:05Z)
• A General Framework for Survival Analysis and Multi-State Modelling [70.31153478610229]
  We use neural ordinary differential equations as a flexible and general method for estimating multi-state survival models. We show that our model exhibits state-of-the-art performance on popular survival data sets and demonstrate its efficacy in a multi-state setting.
  arXiv  Detail & Related papers  (2020-06-08T19:24:54Z)
• Forecasting the Spread of Covid-19 Under Control Scenarios Using LSTM and Dynamic Behavioral Models [2.11622808613962]
  This study proposes a novel hybrid model which combines a Long short-term memory (LSTM) artificial recurrent neural network with dynamic behavioral models. The proposed model considers the effect of multiple factors to enhance the accuracy in predicting the number of cases and deaths across the top ten most-affected countries and Australia.
  arXiv  Detail & Related papers  (2020-05-24T10:43:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.