Training and Evaluating Causal Forecasting Models for Time-Series

• URL: http://arxiv.org/abs/2411.00126v1
• Date: Thu, 31 Oct 2024 18:27:54 GMT
• Title: Training and Evaluating Causal Forecasting Models for Time-Series
• Authors: Thomas Crasson, Yacine Nabet, Mathias Lécuyer,
• Abstract summary: We extend the statistical learning framework to train causal time-series models that generalize better when forecasting the effect of actions outside of their training distribution. We leverage Regression Discontinuity Designs popular in economics to construct a test set of causal treatment effects.
• Score: 1.1218431616419589
• License:
• Abstract: Deep learning time-series models are often used to make forecasts that inform downstream decisions. Since these decisions can differ from those in the training set, there is an implicit requirement that time-series models will generalize outside of their training distribution. Despite this core requirement, time-series models are typically trained and evaluated on in-distribution predictive tasks. We extend the orthogonal statistical learning framework to train causal time-series models that generalize better when forecasting the effect of actions outside of their training distribution. To evaluate these models, we leverage Regression Discontinuity Designs popular in economics to construct a test set of causal treatment effects.

Related papers

• On conditional diffusion models for PDE simulations [53.01911265639582]
  We study score-based diffusion models for forecasting and assimilation of sparse observations. We propose an autoregressive sampling approach that significantly improves performance in forecasting. We also propose a new training strategy for conditional score-based models that achieves stable performance over a range of history lengths.
  arXiv  Detail & Related papers  (2024-10-21T18:31:04Z)
• Zero-shot forecasting of chaotic systems [6.445605125467573]
  Foundation models pre-trained on vast amounts of time-series data from diverse domains. We evaluate whether the zero-shot learning paradigm extends to the challenging task of forecasting chaotic systems.
  arXiv  Detail & Related papers  (2024-09-24T05:56:58Z)
• Implicit Reasoning in Deep Time Series Forecasting [16.750280337155647]
  This work takes an initial step toward assessing the reasoning abilities of deep time series forecasting models. We find that certain linear, patch-based Transformer models generalize effectively in systematically orchestrated out-of-distribution scenarios.
  arXiv  Detail & Related papers  (2024-09-17T02:11:19Z)
• Revisiting SMoE Language Models by Evaluating Inefficiencies with Task Specific Expert Pruning [78.72226641279863]
  Sparse Mixture of Expert (SMoE) models have emerged as a scalable alternative to dense models in language modeling. Our research explores task-specific model pruning to inform decisions about designing SMoE architectures. We introduce an adaptive task-aware pruning technique UNCURL to reduce the number of experts per MoE layer in an offline manner post-training.
  arXiv  Detail & Related papers  (2024-09-02T22:35:03Z)
• On Training Survival Models with Scoring Rules [9.330089124239086]
  This work investigates using scoring rules for model training rather than evaluation. We establish a general framework for training survival models that is model agnostic and can learn event time distributions parametrically or non-parametrically. Empirical comparisons on synthetic and real-world data indicate that scoring rules can be successfully incorporated into model training.
  arXiv  Detail & Related papers  (2024-03-19T20:58:38Z)
• Koopman Ensembles for Probabilistic Time Series Forecasting [6.699751896019971]
  We show that ensembles of independently trained models are highly overconfident and that using a training criterion that explicitly encourages the members to produce predictions with high inter-model variances greatly improves the uncertainty of the ensembles.
  arXiv  Detail & Related papers  (2024-03-11T14:29:56Z)
• Predictive Churn with the Set of Good Models [64.05949860750235]
  We study the effect of conflicting predictions over the set of near-optimal machine learning models. We present theoretical results on the expected churn between models within the Rashomon set. We show how our approach can be used to better anticipate, reduce, and avoid churn in consumer-facing applications.
  arXiv  Detail & Related papers  (2024-02-12T16:15:25Z)
• Attention-Based Ensemble Pooling for Time Series Forecasting [55.2480439325792]
  We propose a method for pooling that performs a weighted average over candidate model forecasts. We test this method on two time-series forecasting problems: multi-step forecasting of the dynamics of the non-stationary Lorenz 63 equation, and one-step forecasting of the weekly incident deaths due to COVID-19.
  arXiv  Detail & Related papers  (2023-10-24T22:59:56Z)
• EAMDrift: An interpretable self retrain model for time series [0.0]
  We present EAMDrift, a novel method that combines forecasts from multiple individual predictors by weighting each prediction according to a performance metric. EAMDrift is designed to automatically adapt to out-of-distribution patterns in data and identify the most appropriate models to use at each moment. Our study on real-world datasets shows that EAMDrift outperforms individual baseline models by 20% and achieves comparable accuracy results to non-interpretable ensemble models.
  arXiv  Detail & Related papers  (2023-05-31T13:25:26Z)
• Model-Attentive Ensemble Learning for Sequence Modeling [86.4785354333566]
  We present Model-Attentive Ensemble learning for Sequence modeling (MAES) MAES is a mixture of time-series experts which leverages an attention-based gating mechanism to specialize the experts on different sequence dynamics and adaptively weight their predictions. We demonstrate that MAES significantly out-performs popular sequence models on datasets subject to temporal shift.
  arXiv  Detail & Related papers  (2021-02-23T05:23:35Z)
• Models, Pixels, and Rewards: Evaluating Design Trade-offs in Visual Model-Based Reinforcement Learning [109.74041512359476]
  We study a number of design decisions for the predictive model in visual MBRL algorithms. We find that a range of design decisions that are often considered crucial, such as the use of latent spaces, have little effect on task performance. We show how this phenomenon is related to exploration and how some of the lower-scoring models on standard benchmarks will perform the same as the best-performing models when trained on the same training data.
  arXiv  Detail & Related papers  (2020-12-08T18:03:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.