SportD: Can VLMs Physically Strategize?

• URL: http://arxiv.org/abs/2607.14616v2
• Date: Sat, 18 Jul 2026 02:57:20 GMT
• Title: SportD: Can VLMs Physically Strategize?
• Abstract summary: SportD is a benchmark comprising 478 on-ball decisions from the 2022 FIFA World Cup.<n>Each model choice is evaluated against a possession-value model that estimates the action that most increases the attacking team's probability of scoring.<n>Across three frontier VLMs, the best selects the highest-valued action on 31.4% of events, compared with 38.9% for the professional players.
• Score: 3.6356011518439684
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Vision--language models have become increasingly capable of interpreting visual scenes, but it remains unclear whether they can use information to make strategically effective decisions. We investigate this question in soccer, where models observe the seconds preceding an on-ball decision and must choose whether to shoot or pass to a specific teammate. Unlike conventional visual-understanding tasks, soccer enables decisions to be evaluated quantitatively by estimating the value of every available action. We introduce SportD, a benchmark comprising 478 on-ball decisions from the 2022 FIFA World Cup. Each model choice is evaluated against a possession-value model that estimates the action that most increases the attacking team's probability of scoring, allowing us to measure both optimal-action accuracy and the value forfeited by suboptimal decisions. Across three frontier VLMs, the best selects the highest-valued action on 31.4% of events, compared with 38.9% for the professional players, and all models incur significantly greater regret. Further analysis reveals a systematic preference for lower-variance and lower-reward actions: VLMs shoot less often and select substantially less progressive passes than either the optimal policy or the real players. The models also reproduce the player's specific action above chance even when that action is suboptimal, suggesting partial imitation of familiar play patterns rather than consistent evaluation of counterfactual alternatives. SportD provides a value-grounded testbed for measuring physical strategic reasoning in VLMs.

Related papers

• SoccerLens: Grounded Soccer Video Understanding Beyond Accuracy [52.09081885158698]
  Vision-language models (VLMs) have recently shown strong potential in soccer video understanding.<n>Existing evaluation protocols focus on classification accuracy and do not assess visual grounding.<n>We introduce SoccerLens, a benchmark for grounded soccer video understanding.
  arXiv  Detail & Related papers  (2026-05-10T15:18:08Z)
• LM Fight Arena: Benchmarking Large Multimodal Models via Game Competition [104.81487689011341]
  We introduce LM Fight Arena, a novel framework that evaluates large multimodal models in Mortal Kombat II.<n>Unlike static evaluations, LM Fight Arena provides a fully automated, reproducible, and objective assessment of an LMM's strategic reasoning capabilities.
  arXiv  Detail & Related papers  (2025-10-10T02:19:21Z)
• What-If Analysis of Large Language Models: Explore the Game World Using Proactive Thinking [50.72154186522052]
  Large language models (LLMs) excel at processing information reactively but lack the ability to systemically explore hypothetical futures.<n>We propose WiA-LLM, a new paradigm that equips LLMs with proactive thinking capabilities.<n>We validate WiA-LLM in Honor of Kings, a complex multiplayer game environment.
  arXiv  Detail & Related papers  (2025-09-05T04:05:27Z)
• Revisiting LLM Value Probing Strategies: Are They Robust and Expressive? [81.49470136653665]
  We evaluate the robustness and expressiveness of value representations across three widely used probing strategies.<n>We show that the demographic context has little effect on the free-text generation, and the models' values only weakly correlate with their preference for value-based actions.
  arXiv  Detail & Related papers  (2025-07-17T18:56:41Z)
• Interpretable Low-Dimensional Modeling of Spatiotemporal Agent States for Decision Making in Football Tactics [0.9207076627649226]
  Rule-based models align with expert knowledge but have not fully considered all players' states.<n>Our approach defines interpretable state variables for both the ball-holder potential pass receivers.<n>The analysis revealed that the distance between the player and the ball, as well as the player's space score, were key factors in determining successful passes.
  arXiv  Detail & Related papers  (2025-06-20T02:37:52Z)
• Are Large Language Models Strategic Decision Makers? A Study of Performance and Bias in Two-Player Non-Zero-Sum Games [56.70628673595041]
  Large Language Models (LLMs) have been increasingly used in real-world settings, yet their strategic decision-making abilities remain largely unexplored. This work investigates the performance and merits of LLMs in canonical game-theoretic two-player non-zero-sum games, Stag Hunt and Prisoner Dilemma. Our structured evaluation of GPT-3.5, GPT-4-Turbo, GPT-4o, and Llama-3-8B shows that these models, when making decisions in these games, are affected by at least one of the following systematic biases.
  arXiv  Detail & Related papers  (2024-07-05T12:30:02Z)
• All by Myself: Learning Individualized Competitive Behaviour with a Contrastive Reinforcement Learning optimization [57.615269148301515]
  In a competitive game scenario, a set of agents have to learn decisions that maximize their goals and minimize their adversaries' goals at the same time. We propose a novel model composed of three neural layers that learn a representation of a competitive game, learn how to map the strategy of specific opponents, and how to disrupt them. Our experiments demonstrate that our model achieves better performance when playing against offline, online, and competitive-specific models, in particular when playing against the same opponent multiple times.
  arXiv  Detail & Related papers  (2023-10-02T08:11:07Z)
• GameEval: Evaluating LLMs on Conversational Games [93.40433639746331]
  We propose GameEval, a novel approach to evaluating large language models (LLMs) GameEval treats LLMs as game players and assigns them distinct roles with specific goals achieved by launching conversations of various forms. We show that GameEval can effectively differentiate the capabilities of various LLMs, providing a comprehensive assessment of their integrated abilities to solve complex problems.
  arXiv  Detail & Related papers  (2023-08-19T14:33:40Z)
• A Strategic Framework for Optimal Decisions in Football 1-vs-1 Shot-Taking Situations: An Integrated Approach of Machine Learning, Theory-Based Modeling, and Game Theory [0.8206877486958002]
  Quantitatively analyzing the strategies involved can provide an objective basis for decision-making. One such critical scenario is shot-taking in football, where decisions, such as whether the attacker should shoot or pass the ball, play a crucial role in the outcome of the game. We propose a novel framework to analyze such scenarios based on game theory, where we estimate the expected payoff with machine learning (ML) models.
  arXiv  Detail & Related papers  (2023-07-27T09:42:25Z)
• Explainable expected goal models for performance analysis in football analytics [5.802346990263708]
  This paper proposes an accurate expected goal model trained consisting of 315,430 shots from seven seasons between 2014-15 and 2020-21 of the top-five European football leagues. To best of our knowledge, this is the first paper that demonstrates a practical application of an explainable artificial intelligence tool aggregated profiles.
  arXiv  Detail & Related papers  (2022-06-14T23:56:03Z)
• Leaving Goals on the Pitch: Evaluating Decision Making in Soccer [21.85419069962932]
  We propose a generic framework to reason about decision-making in soccer by combining techniques from machine learning and artificial intelligence (AI) Our key conclusion is that teams would score more goals if they shot more often from outside the penalty box in a small number of team-specific locations.
  arXiv  Detail & Related papers  (2021-04-07T16:56:31Z)
• Using Player's Body-Orientation to Model Pass Feasibility in Soccer [7.205450793637325]
  Given a monocular video of a soccer match, this paper presents a computational model to estimate the most feasible pass at any given time. The method leverages offensive player's orientation (plus their location) and opponents' spatial configuration to compute the feasibility of pass events within players of the same team. Results show that, by including orientation as a feasibility measure, a robust computational model can be built, reaching more than 0.7 Top-3 accuracy.
  arXiv  Detail & Related papers  (2020-04-15T17:09:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.