Back2Future: Leveraging Backfill Dynamics for Improving Real-time
Predictions in Future
- URL: http://arxiv.org/abs/2106.04420v1
- Date: Tue, 8 Jun 2021 14:48:20 GMT
- Title: Back2Future: Leveraging Backfill Dynamics for Improving Real-time
Predictions in Future
- Authors: Harshavardhan Kamarthi, Alexander Rodr\'iguez, B. Aditya Prakash
- Abstract summary: In real-time forecasting in public health, data collection is a non-trivial and demanding task.
'Backfill' phenomenon and its effect on model performance has been barely studied in the prior literature.
We formulate a novel problem and neural framework Back2Future that aims to refine a given model's predictions in real-time.
- Score: 73.03458424369657
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: In real-time forecasting in public health, data collection is a non-trivial
and demanding task. Often after initially released, it undergoes several
revisions later (maybe due to human or technical constraints) - as a result, it
may take weeks until the data reaches to a stable value. This so-called
'backfill' phenomenon and its effect on model performance has been barely
studied in the prior literature. In this paper, we introduce the multi-variate
backfill problem using COVID-19 as the motivating example. We construct a
detailed dataset composed of relevant signals over the past year of the
pandemic. We then systematically characterize several patterns in backfill
dynamics and leverage our observations for formulating a novel problem and
neural framework Back2Future that aims to refines a given model's predictions
in real-time. Our extensive experiments demonstrate that our method refines the
performance of top models for COVID-19 forecasting, in contrast to non-trivial
baselines, yielding 18% improvement over baselines, enabling us obtain a new
SOTA performance. In addition, we show that our model improves model evaluation
too; hence policy-makers can better understand the true accuracy of forecasting
models in real-time.
Related papers
- On conditional diffusion models for PDE simulations [53.01911265639582]
We study score-based diffusion models for forecasting and assimilation of sparse observations.
We propose an autoregressive sampling approach that significantly improves performance in forecasting.
We also propose a new training strategy for conditional score-based models that achieves stable performance over a range of history lengths.
arXiv Detail & Related papers (2024-10-21T18:31:04Z) - Future-Guided Learning: A Predictive Approach To Enhance Time-Series Forecasting [4.866362841501992]
We introduce Future-Guided Learning, an approach that enhances time-series event forecasting.
Our approach involves two models: a detection model that analyzes future data to identify critical events and a forecasting model that predicts these events based on present data.
When discrepancies arise between the forecasting and detection models, the forecasting model undergoes more substantial updates.
arXiv Detail & Related papers (2024-10-19T21:22:55Z) - Learning Augmentation Policies from A Model Zoo for Time Series Forecasting [58.66211334969299]
We introduce AutoTSAug, a learnable data augmentation method based on reinforcement learning.
By augmenting the marginal samples with a learnable policy, AutoTSAug substantially improves forecasting performance.
arXiv Detail & Related papers (2024-09-10T07:34:19Z) - F-FOMAML: GNN-Enhanced Meta-Learning for Peak Period Demand Forecasting with Proxy Data [65.6499834212641]
We formulate the demand prediction as a meta-learning problem and develop the Feature-based First-Order Model-Agnostic Meta-Learning (F-FOMAML) algorithm.
By considering domain similarities through task-specific metadata, our model improved generalization, where the excess risk decreases as the number of training tasks increases.
Compared to existing state-of-the-art models, our method demonstrates a notable improvement in demand prediction accuracy, reducing the Mean Absolute Error by 26.24% on an internal vending machine dataset and by 1.04% on the publicly accessible JD.com dataset.
arXiv Detail & Related papers (2024-06-23T21:28:50Z) - Predictive Churn with the Set of Good Models [64.05949860750235]
We study the effect of conflicting predictions over the set of near-optimal machine learning models.
We present theoretical results on the expected churn between models within the Rashomon set.
We show how our approach can be used to better anticipate, reduce, and avoid churn in consumer-facing applications.
arXiv Detail & Related papers (2024-02-12T16:15:25Z) - Evaluation of Time-Series Forecasting Models for Chickenpox Cases
Estimation in Hungary [0.0]
We use time-series forecasting techniques to model and predict the future incidence of chickenpox.
We implement and simulate multiple models and data preprocessing techniques on a Hungary-collected dataset.
arXiv Detail & Related papers (2022-09-28T14:27:07Z) - Feature-weighted Stacking for Nonseasonal Time Series Forecasts: A Case
Study of the COVID-19 Epidemic Curves [0.0]
We investigate ensembling techniques in forecasting and examine their potential for use in nonseasonal time-series.
We propose using late data fusion, using a stacked ensemble of two forecasting models and two meta-features that prove their predictive power during a preliminary forecasting stage.
arXiv Detail & Related papers (2021-08-19T14:44:46Z) - Comparing Test Sets with Item Response Theory [53.755064720563]
We evaluate 29 datasets using predictions from 18 pretrained Transformer models on individual test examples.
We find that Quoref, HellaSwag, and MC-TACO are best suited for distinguishing among state-of-the-art models.
We also observe span selection task format, which is used for QA datasets like QAMR or SQuAD2.0, is effective in differentiating between strong and weak models.
arXiv Detail & Related papers (2021-06-01T22:33:53Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.