Kaggle forecasting competitions: An overlooked learning opportunity

• URL: http://arxiv.org/abs/2009.07701v1
• Date: Wed, 16 Sep 2020 14:14:41 GMT
• Title: Kaggle forecasting competitions: An overlooked learning opportunity
• Authors: Casper Solheim Bojer and Jens Peder Meldgaard
• Abstract summary: We review the results from six Kaggle competitions featuring real-life business forecasting tasks. We find that most of the Kaggle datasets are characterized by higher intermittence and entropy than the M-competitions. We find the strong performance of gradient boosted decision trees, increasing success of neural networks for forecasting, and a variety of techniques for adapting machine learning models to the forecasting task.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Competitions play an invaluable role in the field of forecasting, as exemplified through the recent M4 competition. The competition received attention from both academics and practitioners and sparked discussions around the representativeness of the data for business forecasting. Several competitions featuring real-life business forecasting tasks on the Kaggle platform has, however, been largely ignored by the academic community. We believe the learnings from these competitions have much to offer to the forecasting community and provide a review of the results from six Kaggle competitions. We find that most of the Kaggle datasets are characterized by higher intermittence and entropy than the M-competitions and that global ensemble models tend to outperform local single models. Furthermore, we find the strong performance of gradient boosted decision trees, increasing success of neural networks for forecasting, and a variety of techniques for adapting machine learning models to the forecasting task.

Related papers

• Designing Time-Series Models With Hypernetworks & Adversarial Portfolios [0.0]
  This article describes the methods that achieved 4th and 6th place in the forecasting and investment challenges, respectively, of the M6 competition. In the forecasting challenge, we tested a novel meta-learning model that utilizes hypernetworks to design a parametric model tailored to a specific family of forecasting tasks. In the investment challenge, we adjusted portfolio weights to induce greater or smaller correlation between our submission and that of other participants, depending on the current ranking.
  arXiv  Detail & Related papers  (2024-07-29T18:06:29Z)
• A Random Forest-based Prediction Model for Turning Points in Antagonistic Event-Group Competitions [0.0]
  This paper proposes a prediction model based on Random Forest for the turning point of the antagonistic event-group. The model can effectively predict the turning point of the competition situation of the antagonistic event-group.
  arXiv  Detail & Related papers  (2024-05-30T13:13:48Z)
• Predictive Churn with the Set of Good Models [64.05949860750235]
  We study the effect of conflicting predictions over the set of near-optimal machine learning models. We present theoretical results on the expected churn between models within the Rashomon set. We show how our approach can be used to better anticipate, reduce, and avoid churn in consumer-facing applications.
  arXiv  Detail & Related papers  (2024-02-12T16:15:25Z)
• CompeteSMoE -- Effective Training of Sparse Mixture of Experts via Competition [52.2034494666179]
  Sparse mixture of experts (SMoE) offers an appealing solution to scale up the model complexity beyond the mean of increasing the network's depth or width. We propose a competition mechanism to address this fundamental challenge of representation collapse. By routing inputs only to experts with the highest neural response, we show that, under mild assumptions, competition enjoys the same convergence rate as the optimal estimator.
  arXiv  Detail & Related papers  (2024-02-04T15:17:09Z)
• Performative Time-Series Forecasting [71.18553214204978]
  We formalize performative time-series forecasting (PeTS) from a machine-learning perspective. We propose a novel approach, Feature Performative-Shifting (FPS), which leverages the concept of delayed response to anticipate distribution shifts. We conduct comprehensive experiments using multiple time-series models on COVID-19 and traffic forecasting tasks.
  arXiv  Detail & Related papers  (2023-10-09T18:34:29Z)
• Benchmarking Robustness and Generalization in Multi-Agent Systems: A Case Study on Neural MMO [50.58083807719749]
  We present the results of the second Neural MMO challenge, hosted at IJCAI 2022, which received 1600+ submissions. This competition targets robustness and generalization in multi-agent systems. We will open-source our benchmark including the environment wrapper, baselines, a visualization tool, and selected policies for further research.
  arXiv  Detail & Related papers  (2023-08-30T07:16:11Z)
• Improved Bayes Risk Can Yield Reduced Social Welfare Under Competition [99.7047087527422]
  In this work, we demonstrate that competition can fundamentally alter the behavior of machine learning scaling trends. We find many settings where improving data representation quality decreases the overall predictive accuracy across users. At a conceptual level, our work suggests that favorable scaling trends for individual model-providers need not translate to downstream improvements in social welfare.
  arXiv  Detail & Related papers  (2023-06-26T13:06:34Z)
• Approaching sales forecasting using recurrent neural networks and transformers [57.43518732385863]
  We develop three alternatives to tackle the problem of forecasting the customer sales at day/store/item level using deep learning techniques. Our empirical results show how good performance can be achieved by using a simple sequence to sequence architecture with minimal data preprocessing effort. The proposed solution achieves a RMSLE of around 0.54, which is competitive with other more specific solutions to the problem proposed in the Kaggle competition.
  arXiv  Detail & Related papers  (2022-04-16T12:03:52Z)
• Improvements to short-term weather prediction with recurrent-convolutional networks [0.0]
  This paper describes the author's efforts to improve the model further in the second stage of the Weather4cast 2021 competition. The largest quantitative improvements to the competition metrics can be attributed to the increased amount of training data available in the second stage of the competition.
  arXiv  Detail & Related papers  (2021-11-11T14:38:15Z)
• Using Experts' Opinions in Machine Learning Tasks [0.0]
  We propose a general three-step framework for utilizing experts' insights in machine learning tasks. For the case study, we have chosen the task of predicting NCAA Men's Basketball games, which has been the focus of a group of Kaggle competitions. Results suggest that the good performance and high scores of the past models are a result of chance, and not because of a good-performing and stable model.
  arXiv  Detail & Related papers  (2020-08-10T15:48:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.