Scouting By Reward: VLM-TO-IRL-Driven Player Selection For Esports

• URL: http://arxiv.org/abs/2604.14474v1
• Date: Wed, 15 Apr 2026 23:10:49 GMT
• Title: Scouting By Reward: VLM-TO-IRL-Driven Player Selection For Esports
• Abstract summary: We introduce a novel player selection framework that learns professional-specific reward functions from logged gameplay demonstrations.<n>By transitioning from generic skill estimation to scouting "by reward," this framework provides a scalable, workflow-aware digital twin system.
• Score: 8.837655324422192
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Traditional esports scouting workflows rely heavily on manual video review and aggregate performance metrics, which often fail to capture the nuanced decision-making patterns necessary to determine if a prospect fits a specific tactical archetype. To address this, we reframe style-based player evaluation in esports as an Inverse Reinforcement Learning (IRL) problem. In this paper, we introduce a novel player selection framework that learns professional-specific reward functions from logged gameplay demonstrations, allowing organizations to rank candidates by their stylistic alignment with a target star player. Our proposed architecture utilizes a multimodal, two-branch intake: one branch encodes structured state-action trajectories derived from high-resolution in-game telemetry, while the second encodes temporally aligned tactical pseudo-commentary generated by Vision-Language Models (VLMs) from broadcast footage. These representations are fused and evaluated via a Generative Adversarial Imitation Learning (GAIL) objective, where a discriminator learns to capture the unique mechanical and tactical signatures of elite professionals. By transitioning from generic skill estimation to scouting "by reward," this framework provides a scalable, workflow-aware digital twin system that enables data-driven roster construction and targeted talent discovery across massive candidate pools.

Related papers

• Play Like Champions: Counterfactual Feedback Generation in Latent Space [10.090846922590462]
  We introduce Latent Maps of Performance, a framework for counterfactual path generation.<n>We trained a Guided Variational Autoencoder model on 23,305 professional tournament replays.<n>We conclude that there is a trade-off between the path-finding methods we employ and hope that future research will focus on developing model solutions for human improvement.
  arXiv  Detail & Related papers  (2026-06-30T21:15:34Z)
• PEARL: Personalized Streaming Video Understanding Model [50.273809779498464]
  multimodal personalization methods are largely limited to static images or offline videos.<n>PEARL-Bench is the first comprehensive benchmark designed specifically to evaluate this challenging setting.<n>PEARL achieves state-of-the-art performance across 8 offline and online models.
  arXiv  Detail & Related papers  (2026-03-20T18:47:54Z)
• Goal-Driven Reward by Video Diffusion Models for Reinforcement Learning [58.33560203572211]
  We exploit off-the-shelf video diffusion models pretrained on large-scale video datasets.<n>For video-level rewards, we first finetune a pretrained video diffusion model on domain-specific datasets.<n>We then employ a learned forward-backward representation that represents the probability of visiting the goal state from a given state-action pair as frame-level reward.
  arXiv  Detail & Related papers  (2025-11-30T16:22:27Z)
• Learning Skill-Attributes for Transferable Assessment in Video [56.813876909367856]
  Skill assessment from video entails rating the quality of a person's physical performance and explaining what could be done better.<n>Our CrossTrainer approach discovers skill-attributes, such as balance, control, and hand positioning.<n>By abstracting out the shared behaviors indicative of human skill, the proposed video representation generalizes substantially better than an array of existing techniques.
  arXiv  Detail & Related papers  (2025-11-17T23:53:06Z)
• X-Ego: Acquiring Team-Level Tactical Situational Awareness via Cross-Egocentric Contrastive Video Representation Learning [1.1765015608581086]
  We introduce X-Ego-CS, a benchmark dataset consisting of 124 hours of gameplay footage from 45 professional-level matches of the popular e-sports game Counter-Strike 2.<n>X-Ego-CS provides cross-egocentric video streams that synchronously capture all players' first-person perspectives along with state-action trajectories.<n>We propose Cross-Ego Contrastive Learning ( CECL), which aligns teammates' egocentric visual streams to foster team-level situational awareness from an individual's perspective.
  arXiv  Detail & Related papers  (2025-10-22T00:48:35Z)
• AVA: Attentive VLM Agent for Mastering StarCraft II [56.07921367623274]
  We introduce Attentive VLM Agent (AVA), a multimodal StarCraft II agent that aligns artificial agent perception with the human gameplay experience.<n>Our agent addresses this limitation by incorporating RGB visual inputs and natural language observations that more closely simulate human cognitive processes during gameplay.
  arXiv  Detail & Related papers  (2025-03-07T12:54:25Z)
• ExpertAF: Expert Actionable Feedback from Video [81.46431188306397]
  We introduce a novel method to generate actionable feedback from video of a person doing a physical activity, such as basketball or soccer.<n>Our method takes a video demonstration and its accompanying 3D body pose and generates expert commentary describing what the person is doing well and what they could improve.<n>We show how to leverage Ego-Exo4D's [29] videos of skilled activity and expert commentary together with a strong language model to create a weakly-supervised training dataset for this task.
  arXiv  Detail & Related papers  (2024-08-01T16:13:07Z)
• Advanced Volleyball Stats for All Levels: Automatic Setting Tactic Detection and Classification with a Single Camera [15.032833555418314]
  We present two novel end-to-end computer vision frameworks designed specifically for setting strategy classification in volleyball matches from a single camera view. Our frameworks combine setting ball trajectory recognition with a novel set trajectory classifier to generate comprehensive and advanced statistical data. Our system allows for real-time deployment, enabling in-game strategy analysis and on-the-spot gameplan adjustments.
  arXiv  Detail & Related papers  (2023-09-26T08:29:02Z)
• Soft Expert Reward Learning for Vision-and-Language Navigation [94.86954695912125]
  Vision-and-Language Navigation (VLN) requires an agent to find a specified spot in an unseen environment by following natural language instructions. We introduce a Soft Expert Reward Learning (SERL) model to overcome the reward engineering designing and generalisation problems of the VLN task.
  arXiv  Detail & Related papers  (2020-07-21T14:17:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.