Do global forecasting models require frequent retraining?

• URL: http://arxiv.org/abs/2505.00356v1
• Date: Thu, 01 May 2025 07:00:29 GMT
• Title: Do global forecasting models require frequent retraining?
• Authors: Marco Zanotti,
• Abstract summary: We show that less frequent retraining strategies maintain the forecast accuracy while reducing the computational costs.<n>We also found that machine learning models are a marginally better choice to reduce the costs of forecasting when coupled with less frequent model retraining strategies.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In an era of increasing computational capabilities and growing environmental consciousness, organizations face a critical challenge in balancing the accuracy of forecasting models with computational efficiency and sustainability. Global forecasting models, lowering the computational time, have gained significant attention over the years. However, the common practice of retraining these models with new observations raises important questions about the costs of forecasting. Using ten different machine learning and deep learning models, we analyzed various retraining scenarios, ranging from continuous updates to no retraining at all, across two large retail datasets. We showed that less frequent retraining strategies maintain the forecast accuracy while reducing the computational costs, providing a more sustainable approach to large-scale forecasting. We also found that machine learning models are a marginally better choice to reduce the costs of forecasting when coupled with less frequent model retraining strategies as the frequency of the data increases. Our findings challenge the conventional belief that frequent retraining is essential for maintaining forecasting accuracy. Instead, periodic retraining offers a good balance between predictive performance and efficiency, both in the case of point and probabilistic forecasting. These insights provide actionable guidelines for organizations seeking to optimize forecasting pipelines while reducing costs and energy consumption.

Related papers

• Future-Guided Learning: A Predictive Approach To Enhance Time-Series Forecasting [4.866362841501992]
  We introduce Future-Guided Learning, an approach that enhances time-series event forecasting through a dynamic feedback mechanism inspired by predictive coding.<n>Our method involves two models: a detection model that analyzes future data to identify critical events and a forecasting model that predicts these events based on current data.<n>We validate our approach on a variety of tasks, demonstrating a 44.8% increase in AUC-ROC for seizure prediction using EEG data, and a 48.7% reduction in MSE for forecasting in nonlinear dynamical systems.
  arXiv  Detail & Related papers  (2024-10-19T21:22:55Z)
• Automated Data Augmentation for Few-Shot Time Series Forecasting: A Reinforcement Learning Approach Guided by a Model Zoo [34.40047933452929]
  We present a pilot study on using reinforcement learning (RL) for time series data augmentation.<n>Our method, ReAugment, tackles three critical questions: which parts of the training set should be augmented, how the augmentation should be performed, and what advantages RL brings to the process.
  arXiv  Detail & Related papers  (2024-09-10T07:34:19Z)
• Integrating the Expected Future in Load Forecasts with Contextually Enhanced Transformer Models [4.868683179698627]
  This work conceptualizes forecasting as a sequence-to-sequence prediction problem. We introduce contextually-enhanced transformer models designed to leverage all contextual information effectively. Compared to other state-of-the-art methods, our approach consistently outperforms existing models.
  arXiv  Detail & Related papers  (2024-08-26T12:32:40Z)
• Forecast-PEFT: Parameter-Efficient Fine-Tuning for Pre-trained Motion Forecasting Models [68.23649978697027]
  Forecast-PEFT is a fine-tuning strategy that freezes the majority of the model's parameters, focusing adjustments on newly introduced prompts and adapters. Our experiments show that Forecast-PEFT outperforms traditional full fine-tuning methods in motion prediction tasks. Forecast-FT further improves prediction performance, evidencing up to a 9.6% enhancement over conventional baseline methods.
  arXiv  Detail & Related papers  (2024-07-28T19:18:59Z)
• Improving Sequential Market Coordination via Value-oriented Renewable Energy Forecasting [3.0665531066360066]
  We propose using a trained forecasting model, referred to as value-oriented forecasting, to determine RES Improved Entering Quantities (RIEQ) more efficiently during the operational phase.<n>Unlike traditional models that minimize statistical forecasting errors, our approach trains model parameters to minimize the expected overall operating costs across both DA and RT markets.
  arXiv  Detail & Related papers  (2024-05-15T00:04:08Z)
• Impact of data for forecasting on performance of model predictive control in buildings with smart energy storage [0.0]
  The impact on forecast accuracy of measures to improve model data efficiency are quantified. The use of more than 2 years of training data for load prediction models provided no significant improvement in forecast accuracy. Reused models and those trained with 3 months of data had on average 10% higher error than baseline, indicating that deploying MPC systems without prior data collection may be economic.
  arXiv  Detail & Related papers  (2024-02-19T21:01:11Z)
• Weather Prediction with Diffusion Guided by Realistic Forecast Processes [49.07556359513563]
  We introduce a novel method that applies diffusion models (DM) for weather forecasting. Our method can achieve both direct and iterative forecasting with the same modeling framework. The flexibility and controllability of our model empowers a more trustworthy DL system for the general weather community.
  arXiv  Detail & Related papers  (2024-02-06T21:28:42Z)
• ExtremeCast: Boosting Extreme Value Prediction for Global Weather Forecast [57.6987191099507]
  We introduce Exloss, a novel loss function that performs asymmetric optimization and highlights extreme values to obtain accurate extreme weather forecast. We also introduce ExBooster, which captures the uncertainty in prediction outcomes by employing multiple random samples. Our solution can achieve state-of-the-art performance in extreme weather prediction, while maintaining the overall forecast accuracy comparable to the top medium-range forecast models.
  arXiv  Detail & Related papers  (2024-02-02T10:34:13Z)
• Temporal Knowledge Distillation for Time-Sensitive Financial Services Applications [7.1795069620810805]
  Anomaly detection is frequently used in key compliance and risk functions such as financial crime detection fraud and cybersecurity. Keeping up with the rapid changes by retraining the models with the latest data patterns introduces pressures in balancing the historical and current patterns. The proposed approach provides advantages in retraining times while improving the model performance.
  arXiv  Detail & Related papers  (2023-12-28T03:04:30Z)
• Global Neural Networks and The Data Scaling Effect in Financial Time Series Forecasting [5.299784478982814]
  We demonstrate that the conventional practice of estimating models locally in data-scarce environments may underlie the mixed empirical performance observed in prior work.<n>We employ a dataset comprising over 10,000 global stocks and implement a global estimation strategy that pools information across cross-sections.
  arXiv  Detail & Related papers  (2023-09-05T09:18:45Z)
• Mlinear: Rethink the Linear Model for Time-series Forecasting [9.841293660201261]
  Mlinear is a simple yet effective method based mainly on linear layers. We introduce a new loss function that significantly outperforms the widely used mean squared error (MSE) on multiple datasets. Our method significantly outperforms PatchTST with a ratio of 21:3 at 336 sequence length input and 29:10 at 512 sequence length input.
  arXiv  Detail & Related papers  (2023-05-08T15:54:18Z)
• Learning Sample Difficulty from Pre-trained Models for Reliable Prediction [55.77136037458667]
  We propose to utilize large-scale pre-trained models to guide downstream model training with sample difficulty-aware entropy regularization. We simultaneously improve accuracy and uncertainty calibration across challenging benchmarks.
  arXiv  Detail & Related papers  (2023-04-20T07:29:23Z)
• Uncertainty-Aware Time-to-Event Prediction using Deep Kernel Accelerated Failure Time Models [11.171712535005357]
  We propose Deep Kernel Accelerated Failure Time models for the time-to-event prediction task. Our model shows better point estimate performance than recurrent neural network based baselines in experiments on two real-world datasets.
  arXiv  Detail & Related papers  (2021-07-26T14:55:02Z)
• Back2Future: Leveraging Backfill Dynamics for Improving Real-time Predictions in Future [73.03458424369657]
  In real-time forecasting in public health, data collection is a non-trivial and demanding task. 'Backfill' phenomenon and its effect on model performance has been barely studied in the prior literature. We formulate a novel problem and neural framework Back2Future that aims to refine a given model's predictions in real-time.
  arXiv  Detail & Related papers  (2021-06-08T14:48:20Z)
• Learning Interpretable Deep State Space Model for Probabilistic Time Series Forecasting [98.57851612518758]
  Probabilistic time series forecasting involves estimating the distribution of future based on its history. We propose a deep state space model for probabilistic time series forecasting whereby the non-linear emission model and transition model are parameterized by networks. We show in experiments that our model produces accurate and sharp probabilistic forecasts.
  arXiv  Detail & Related papers  (2021-01-31T06:49:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.