Using Experts' Opinions in Machine Learning Tasks

• URL: http://arxiv.org/abs/2008.04216v3
• Date: Fri, 3 Dec 2021 18:38:32 GMT
• Title: Using Experts' Opinions in Machine Learning Tasks
• Authors: Jafar Habibi, Amir Fazelinia, Issa Annamoradnejad
• Abstract summary: We propose a general three-step framework for utilizing experts' insights in machine learning tasks. For the case study, we have chosen the task of predicting NCAA Men's Basketball games, which has been the focus of a group of Kaggle competitions. Results suggest that the good performance and high scores of the past models are a result of chance, and not because of a good-performing and stable model.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In machine learning tasks, especially in the tasks of prediction, scientists tend to rely solely on available historical data and disregard unproven insights, such as experts' opinions, polls, and betting odds. In this paper, we propose a general three-step framework for utilizing experts' insights in machine learning tasks and build four concrete models for a sports game prediction case study. For the case study, we have chosen the task of predicting NCAA Men's Basketball games, which has been the focus of a group of Kaggle competitions in recent years. Results highly suggest that the good performance and high scores of the past models are a result of chance, and not because of a good-performing and stable model. Furthermore, our proposed models can achieve more steady results with lower log loss average (best at 0.489) compared to the top solutions of the 2019 competition (>0.503), and reach the top 1%, 10% and 1% in the 2017, 2018 and 2019 leaderboards, respectively.

Related papers

• Zero-shot Retrieval: Augmenting Pre-trained Models with Search Engines [83.65380507372483]
  Large pre-trained models can dramatically reduce the amount of task-specific data required to solve a problem, but they often fail to capture domain-specific nuances out of the box. This paper shows how to leverage recent advances in NLP and multi-modal learning to augment a pre-trained model with search engine retrieval.
  arXiv  Detail & Related papers  (2023-11-29T05:33:28Z)
• Evaluating Soccer Match Prediction Models: A Deep Learning Approach and Feature Optimization for Gradient-Boosted Trees [0.8009842832476994]
  The 2023 Soccer Prediction Challenge required the prediction of match results first in terms of the exact goals scored by each team, and second, in terms of the probabilities for a win, draw, and loss. A CatBoost model was employed using pi-ratings as the features, which were initially identified as the optimal choice for calculating the win/draw/loss probabilities. In this study, we aimed to assess the performance of a deep learning model and determine the optimal feature set for a gradient-boosted tree model.
  arXiv  Detail & Related papers  (2023-09-26T10:05:46Z)
• Supervised Learning for Table Tennis Match Prediction [2.7835697868135902]
  This paper proposes the use of machine learning to predict the outcome of table tennis single matches. We use player and match statistics as features and evaluate their relative importance in an ablation study. The results can serve as a baseline for future table tennis prediction models, and can feed back to prediction research in similar ball sports.
  arXiv  Detail & Related papers  (2023-03-28T17:42:13Z)
• Machine learning for sports betting: should model selection be based on accuracy or calibration? [0.0]
  We train models on NBA data over several seasons and run betting experiments on a single season. We show that using calibration, rather than accuracy, as the basis for model selection leads to greater returns.
  arXiv  Detail & Related papers  (2023-03-10T16:22:38Z)
• Synthetic Model Combination: An Instance-wise Approach to Unsupervised Ensemble Learning [92.89846887298852]
  Consider making a prediction over new test data without any opportunity to learn from a training set of labelled data. Give access to a set of expert models and their predictions alongside some limited information about the dataset used to train them.
  arXiv  Detail & Related papers  (2022-10-11T10:20:31Z)
• GCN-WP -- Semi-Supervised Graph Convolutional Networks for Win Prediction in Esports [84.55775845090542]
  We propose a semi-supervised win prediction model for esports based on graph convolutional networks. GCN-WP integrates over 30 features about the match and players and employs graph convolution to classify games based on their neighborhood. Our model achieves state-of-the-art prediction accuracy when compared to machine learning or skill rating models for LoL.
  arXiv  Detail & Related papers  (2022-07-26T21:38:07Z)
• Exploring Strategies for Generalizable Commonsense Reasoning with Pre-trained Models [62.28551903638434]
  We measure the impact of three different adaptation methods on the generalization and accuracy of models. Experiments with two models show that fine-tuning performs best, by learning both the content and the structure of the task, but suffers from overfitting and limited generalization to novel answers. We observe that alternative adaptation methods like prefix-tuning have comparable accuracy, but generalize better to unseen answers and are more robust to adversarial splits.
  arXiv  Detail & Related papers  (2021-09-07T03:13:06Z)
• Kaggle forecasting competitions: An overlooked learning opportunity [0.0]
  We review the results from six Kaggle competitions featuring real-life business forecasting tasks. We find that most of the Kaggle datasets are characterized by higher intermittence and entropy than the M-competitions. We find the strong performance of gradient boosted decision trees, increasing success of neural networks for forecasting, and a variety of techniques for adapting machine learning models to the forecasting task.
  arXiv  Detail & Related papers  (2020-09-16T14:14:41Z)
• Interpretable Real-Time Win Prediction for Honor of Kings, a Popular Mobile MOBA Esport [51.20042288437171]
  We propose a Two-Stage Spatial-Temporal Network (TSSTN) that can provide accurate real-time win predictions. Experiment results and applications in real-world live streaming scenarios showed that the proposed TSSTN model is effective both in prediction accuracy and interpretability.
  arXiv  Detail & Related papers  (2020-08-14T12:00:58Z)
• Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuning [134.15174177472807]
  We introduce adversarial training into self-supervision, to provide general-purpose robust pre-trained models for the first time. We conduct extensive experiments to demonstrate that the proposed framework achieves large performance margins.
  arXiv  Detail & Related papers  (2020-03-28T18:28:33Z)
• Descriptive and Predictive Analysis of Euroleague Basketball Games and the Wisdom of Basketball Crowds [0.0]
  This study focuses on the prediction of basketball games in the Euroleague competition using machine learning modelling. We find that simple machine learning models give accuracy not greater than 67% on the test set, worse than some sophisticated benchmark models. We argue why the accuracy level of this group of "experts" should be set as the benchmark for future studies in the prediction of (European) basketball games using machine learning.
  arXiv  Detail & Related papers  (2020-02-19T22:04:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.