WorldCupArena: Fine-Grained Evaluation of Language Models and Deep-Research Agents on Football Forecasting

• URL: http://arxiv.org/abs/2607.18084v1
• Date: Mon, 20 Jul 2026 15:52:48 GMT
• Title: WorldCupArena: Fine-Grained Evaluation of Language Models and Deep-Research Agents on Football Forecasting
• Abstract summary: We present WorldCupArena, a benchmark for language models and deep-research agents.<n>It predicts the result and score, likely players and events, match statistics, and the outcome of the competition.<n>We report result accuracy, exact-score accuracy, and a scoreline score that gives some credit when a predicted score is close but not exact.
• Score: 10.941871398755403
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Predicting a football match before kickoff requires more than knowing past results: a model must use changing information and make a clear prediction before the answer is available. We present WorldCupArena, a dynamic benchmark for language models and deep-research agents. The 2026 FIFA World Cup is its first evaluation, and the same process can be reused for future leagues and cups. Before each match, a model either receives a common evidence package or searches for information itself. It predicts the result and score, likely players and events, match statistics, and the outcome of the competition. After the match, these predictions are compared with the recorded result. We report result accuracy, exact-score accuracy, and a scoreline score that gives some credit when a predicted score is close but not exact, together with scores for the other prediction tasks. Across 104 matches and 13 systems, models with similar result accuracy differ more clearly on detailed predictions. Compared with betting-market and human-fan baselines, the best system shows only small gains in result and exact-score accuracy, but a clearer gain in Scoreline. New schedules can be added as they begin, allowing the benchmark to evaluate future models without using outcomes that are already known. Code, prompts, predictions, and evaluation scripts are open sourced at https://github.com/wzk1015/WorldCupArena.

Related papers

• Forecasting the Winner of a Live Tennis Match [2.9832083520093353]
  This study examines how pre-match and live information can be most effectively integrated into a model to produce accurate win-probability estimates.<n>The analysis uses 8,222 Grand Slam matches containing a total of 1,505,355 points.
  arXiv  Detail & Related papers  (2026-09-07T15:21:58Z)
• AI World Cup 2026: Benchmarking Large Language Models for End-to-End Football Tournament Prediction [0.0]
  This paper reports the completed emphAI World Cup benchmark, in which ten-based assistants made a single pre-tournament forecast of the entire 2026 FIFA World Cup.<n>Forecasts covered group-stage scores, group rankings, the knockout bracket, final placings, confidence values, and short explanations.<n>GPT-5.5 Thinking finished first with 744 points, followed by GPT-5.5 with 717, Gemini with 699, and Qwen 3.7 with 687.
  arXiv  Detail & Related papers  (2026-08-04T10:07:14Z)
• Scaling Open-Ended Reasoning to Predict the Future [56.672065928345525]
  We train language models to make predictions on open-ended forecasting questions.<n>To scale up training data, we synthesize novel forecasting questions from global events reported in daily news.<n>We find calibration improvements from forecasting training generalize across popular benchmarks.
  arXiv  Detail & Related papers  (2025-12-31T18:59:51Z)
• Consistency Checks for Language Model Forecasters [54.62507816753479]
  We measure the performance of forecasters in terms of the consistency of their predictions on different logically-related questions.<n>We build an automated evaluation system that generates a set of base questions, instantiates consistency checks from these questions, elicits predictions of the forecaster, and measures the consistency of the predictions.
  arXiv  Detail & Related papers  (2024-12-24T16:51:35Z)
• Bayes-xG: Player and Position Correction on Expected Goals (xG) using Bayesian Hierarchical Approach [55.2480439325792]
  This study investigates the influence of player or positional factors in predicting a shot resulting in a goal, measured by the expected goals (xG) metric. It uses publicly available data from StatsBomb to analyse 10,000 shots from the English Premier League. The study extends its analysis to data from Spain's La Liga and Germany's Bundesliga, yielding comparable results.
  arXiv  Detail & Related papers  (2023-11-22T21:54:02Z)
• Evaluating Soccer Match Prediction Models: A Deep Learning Approach and Feature Optimization for Gradient-Boosted Trees [0.8009842832476994]
  The 2023 Soccer Prediction Challenge required the prediction of match results first in terms of the exact goals scored by each team, and second, in terms of the probabilities for a win, draw, and loss. A CatBoost model was employed using pi-ratings as the features, which were initially identified as the optimal choice for calculating the win/draw/loss probabilities. In this study, we aimed to assess the performance of a deep learning model and determine the optimal feature set for a gradient-boosted tree model.
  arXiv  Detail & Related papers  (2023-09-26T10:05:46Z)
• Predicting Football Match Outcomes with eXplainable Machine Learning and the Kelly Index [0.0]
  A machine learning approach is developed for predicting the outcomes of football matches. The dataset originated from the Premier League match data covering the 2019-2021 seasons. The paper also devised an investment strategy in order to evaluate its effectiveness by benchmarking against bookmaker odds.
  arXiv  Detail & Related papers  (2022-11-28T19:32:58Z)
• Betting the system: Using lineups to predict football scores [0.0]
  This paper aims to reduce randomness in football by analysing the role of lineups in final scores. Football clubs invest millions of dollars on lineups and knowing how individual statistics translate to better outcomes can optimise investments. Sports betting is growing exponentially and being able to predict the future is profitable and desirable.
  arXiv  Detail & Related papers  (2022-10-12T15:47:42Z)
• GCN-WP -- Semi-Supervised Graph Convolutional Networks for Win Prediction in Esports [84.55775845090542]
  We propose a semi-supervised win prediction model for esports based on graph convolutional networks. GCN-WP integrates over 30 features about the match and players and employs graph convolution to classify games based on their neighborhood. Our model achieves state-of-the-art prediction accuracy when compared to machine learning or skill rating models for LoL.
  arXiv  Detail & Related papers  (2022-07-26T21:38:07Z)
• Evaluation of soccer team defense based on prediction models of ball recovery and being attacked [0.8921166277011345]
  We propose a method to evaluate team defense based on the prediction of ball recovery and being attacked. Using data from 45 soccer matches, we examined the relationship between the proposed index and team performance.
  arXiv  Detail & Related papers  (2021-03-17T13:15:41Z)
• Interpretable Real-Time Win Prediction for Honor of Kings, a Popular Mobile MOBA Esport [51.20042288437171]
  We propose a Two-Stage Spatial-Temporal Network (TSSTN) that can provide accurate real-time win predictions. Experiment results and applications in real-world live streaming scenarios showed that the proposed TSSTN model is effective both in prediction accuracy and interpretability.
  arXiv  Detail & Related papers  (2020-08-14T12:00:58Z)
• Evaluation Toolkit For Robustness Testing Of Automatic Essay Scoring Systems [64.4896118325552]
  We evaluate the current state-of-the-art AES models using a model adversarial evaluation scheme and associated metrics. We find that AES models are highly overstable. Even heavy modifications(as much as 25%) with content unrelated to the topic of the questions do not decrease the score produced by the models.
  arXiv  Detail & Related papers  (2020-07-14T03:49:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.