The cost of ensembling: is it always worth combining?

• URL: http://arxiv.org/abs/2506.04677v2
• Date: Wed, 09 Jul 2025 12:32:09 GMT
• Title: The cost of ensembling: is it always worth combining?
• Authors: Marco Zanotti,
• Abstract summary: Trade-off between forecast accuracy and computational cost is emerging as an extremely relevant topic.<n>We evaluated ten base models and eight ensemble configurations across two large-scale retail datasets.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Given the continuous increase in dataset sizes and the complexity of forecasting models, the trade-off between forecast accuracy and computational cost is emerging as an extremely relevant topic, especially in the context of ensemble learning for time series forecasting. To asses it, we evaluated ten base models and eight ensemble configurations across two large-scale retail datasets (M5 and VN1), considering both point and probabilistic accuracy under varying retraining frequencies. We showed that ensembles consistently improve forecasting performance, particularly in probabilistic settings. However, these gains come at a substantial computational cost, especially for larger, accuracy-driven ensembles. We found that reducing retraining frequency significantly lowers costs, with minimal impact on accuracy, particularly for point forecasts. Moreover, efficiency-driven ensembles offer a strong balance, achieving competitive accuracy with considerably lower costs compared to accuracy-optimized combinations. Most importantly, small ensembles of two or three models are often sufficient to achieve near-optimal results. These findings provide practical guidelines for deploying scalable and cost-efficient forecasting systems, supporting the broader goals of sustainable AI in forecasting. Overall, this work shows that careful ensemble design and retraining strategy selection can yield accurate, robust, and cost-effective forecasts suitable for real-world applications.

Related papers

• Comparative Analysis of Modern Machine Learning Models for Retail Sales Forecasting [0.0]
  When forecasts underestimate the level of sales, firms experience lost sales, shortages, and impact on the reputation of the retailer in their relevant market.<n>This study provides an exhaustive assessment of the forecasting models applied to a high-resolution brick-and-mortar retail dataset.
  arXiv  Detail & Related papers  (2025-06-06T10:08:17Z)
• Do global forecasting models require frequent retraining? [0.0]
  We show that less frequent retraining strategies maintain the forecast accuracy while reducing the computational costs.<n>We also found that machine learning models are a marginally better choice to reduce the costs of forecasting when coupled with less frequent model retraining strategies.
  arXiv  Detail & Related papers  (2025-05-01T07:00:29Z)
• Forecasting Company Fundamentals [19.363166648866066]
  We evaluate 24 deterministic and probabilistic company fundamentals forecasting models on real company data.<n>We find that deep learning models provide superior forecasting performance to classical models.<n>We show how these high-quality forecasts can benefit automated stock allocation.
  arXiv  Detail & Related papers  (2024-10-21T14:21:43Z)
• Sparse Mixture-of-Experts for Compositional Generalization: Empirical Evidence and Theoretical Foundations of Optimal Sparsity [89.81738321188391]
  This study investigates the relationship between task complexity and optimal sparsity in SMoE models.<n>We show that the optimal sparsity lies between minimal activation (1-2 experts) and full activation, with the exact number scaling proportionally to task complexity.
  arXiv  Detail & Related papers  (2024-10-17T18:40:48Z)
• Learning Graph Structures and Uncertainty for Accurate and Calibrated Time-series Forecasting [65.40983982856056]
  We introduce STOIC, that leverages correlations between time-series to learn underlying structure between time-series and to provide well-calibrated and accurate forecasts. Over a wide-range of benchmark datasets STOIC provides 16% more accurate and better-calibrated forecasts.
  arXiv  Detail & Related papers  (2024-07-02T20:14:32Z)
• When Rigidity Hurts: Soft Consistency Regularization for Probabilistic Hierarchical Time Series Forecasting [69.30930115236228]
  Probabilistic hierarchical time-series forecasting is an important variant of time-series forecasting. Most methods focus on point predictions and do not provide well-calibrated probabilistic forecasts distributions. We propose PROFHiT, a fully probabilistic hierarchical forecasting model that jointly models forecast distribution of entire hierarchy.
  arXiv  Detail & Related papers  (2023-10-17T20:30:16Z)
• Ensemble Modeling for Time Series Forecasting: an Adaptive Robust Optimization Approach [3.7565501074323224]
  This paper proposes a new methodology for building robust ensembles of time series forecasting models. We demonstrate the effectiveness of our method through a series of synthetic experiments and real-world applications.
  arXiv  Detail & Related papers  (2023-04-09T20:30:10Z)
• When Rigidity Hurts: Soft Consistency Regularization for Probabilistic Hierarchical Time Series Forecasting [69.30930115236228]
  Probabilistic hierarchical time-series forecasting is an important variant of time-series forecasting. Most methods focus on point predictions and do not provide well-calibrated probabilistic forecasts distributions. We propose PROFHiT, a fully probabilistic hierarchical forecasting model that jointly models forecast distribution of entire hierarchy.
  arXiv  Detail & Related papers  (2022-06-16T06:13:53Z)
• Functional Ensemble Distillation [18.34081591772928]
  We investigate how to best distill an ensemble's predictions using an efficient model. We find that learning the distilled model via a simple augmentation scheme in the form of mixup augmentation significantly boosts the performance.
  arXiv  Detail & Related papers  (2022-06-05T14:07:17Z)
• Sparse MoEs meet Efficient Ensembles [49.313497379189315]
  We study the interplay of two popular classes of such models: ensembles of neural networks and sparse mixture of experts (sparse MoEs) We present Efficient Ensemble of Experts (E$3$), a scalable and simple ensemble of sparse MoEs that takes the best of both classes of models, while using up to 45% fewer FLOPs than a deep ensemble.
  arXiv  Detail & Related papers  (2021-10-07T11:58:35Z)
• Generalizable Mixed-Precision Quantization via Attribution Rank Preservation [90.26603048354575]
  We propose a generalizable mixed-precision quantization (GMPQ) method for efficient inference. Our method obtains competitive accuracy-complexity trade-off compared with the state-of-the-art mixed-precision networks.
  arXiv  Detail & Related papers  (2021-08-05T16:41:57Z)
• Deep Learning for Post-Processing Ensemble Weather Forecasts [14.622977874836298]
  We propose a mixed model that uses only a subset of the original weather trajectories combined with a post-processing step using deep neural networks. We show that our post-processing can use fewer trajectories to achieve comparable results to the full ensemble.
  arXiv  Detail & Related papers  (2020-05-18T14:23:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.