DIAMOND: An LLM-Driven Agent for Context-Aware Baseball Highlight Summarization

• URL: http://arxiv.org/abs/2506.02351v1
• Date: Tue, 03 Jun 2025 01:10:20 GMT
• Title: DIAMOND: An LLM-Driven Agent for Context-Aware Baseball Highlight Summarization
• Abstract summary: We introduce DIAMOND, an agent for context-aware baseball highlight summarization.<n>We use structured sports analytics and natural language reasoning to quantify play importance.<n>Our results highlight the potential of modular, interpretable agent-based frameworks for event-level summarization.
• Score: 9.67464173044675
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Traditional approaches -- such as Win Probability Added (WPA)-based ranking or computer vision-driven event detection -- can identify scoring plays but often miss strategic depth, momentum shifts, and storyline progression. Manual curation remains the gold standard but is resource-intensive and not scalable. We introduce DIAMOND, an LLM-driven agent for context-aware baseball highlight summarization that integrates structured sports analytics with natural language reasoning. DIAMOND leverages sabermetric features -- Win Expectancy, WPA, and Leverage Index -- to quantify play importance, while an LLM module enhances selection based on contextual narrative value. This hybrid approach ensures both quantitative rigor and qualitative richness, surpassing the limitations of purely statistical or vision-based systems. Evaluated on five diverse Korean Baseball Organization League games, DIAMOND improves F1-score from 42.9% (WPA-only) to 84.8%, outperforming both commercial and statistical baselines. Though limited in scale, our results highlight the potential of modular, interpretable agent-based frameworks for event-level summarization in sports and beyond.

Related papers

• SUMMIR: A Hallucination-Aware Framework for Ranking Sports Insights from LLMs [15.398772820579204]
  In this paper, we address the task of automatically extracting meaningful pre-game and post-game insights from articles published before and after matches.<n>We curate a dataset of 7,900 news articles covering 800 matches across four major sports: Cricket, Soccer, Basketball, and Baseball.<n>Our results demonstrate the effectiveness of this approach in generating high-quality, relevant insights, while also revealing significant differences in factual consistency and interestingness across LLMs.
  arXiv  Detail & Related papers  (2026-03-30T12:45:15Z)
• Real-time Win Probability and Latent Player Ability via STATS X in Team Sports [0.0]
  This study proposes a statistically grounded framework for real-time win probability evaluation and player assessment in team sports.<n>We introduce a continuous dominance indicator (T-score) that maps final scores to real values consistent with win/lose outcomes.<n>We also define a latent contribution index, STATS X, which quantifies a player's involvement in favorable dominance intervals.
  arXiv  Detail & Related papers  (2026-02-23T05:00:44Z)
• BotzoneBench: Scalable LLM Evaluation via Graded AI Anchors [9.224594551677374]
  Large Language Models (LLMs) are increasingly deployed in interactive environments requiring strategic decision-making.<n>Recent game-based evaluations employ LLM-vs-LLM tournaments that produce relative rankings dependent on transient model pools.<n>Here we show that anchoring LLM evaluation to fixed hierarchies of skill-calibrated game Artificial Intelligence (AI) enables linear-time absolute skill measurement.
  arXiv  Detail & Related papers  (2026-01-22T13:15:08Z)
• BEAR: Benchmarking and Enhancing Multimodal Language Models for Atomic Embodied Capabilities [61.173773299032746]
  Embodied capabilities refer to a suite of fundamental abilities for an agent to perceive, comprehend, and interact with the physical world.<n>We introduce BEAR, a comprehensive and fine-grained benchmark that evaluates MLLMs on atomic embodied capabilities.<n> BEAR comprises 4,469 interleaved image-video-text entries across 14 domains in 6 categories, including tasks from low-level pointing, trajectory understanding, spatial reasoning, to high-level planning.<n>We propose BEAR-Agent, a multimodal conversable agent that integrates pretrained vision models to strengthen MLLM perception, 3D understanding, and planning capabilities.
  arXiv  Detail & Related papers  (2025-10-09T19:18:36Z)
• LLMsPark: A Benchmark for Evaluating Large Language Models in Strategic Gaming Contexts [19.97430860742638]
  We present a game theory-based evaluation platform that measures large language models' decision-making strategies and social behaviors in classic game-theoretic settings.<n>Our system cross-evaluates 15 leading LLMs using leaderboard rankings and scoring mechanisms.<n>This work introduces a novel perspective for evaluating LLMs' strategic intelligence, enriching existing benchmarks and broadening their assessment in interactive, game-theoretic scenarios.
  arXiv  Detail & Related papers  (2025-09-20T10:21:17Z)
• SKATE, a Scalable Tournament Eval: Weaker LLMs differentiate between stronger ones using verifiable challenges [2.184775414778289]
  We introduce SKATE: a novel evaluation framework in which large language models (LLMs) compete by generating verifiable tasks for one another.<n>Our core is to treat evaluation as a game: models as both task-setters and solvers, incentivized to create questions which highlight their own strengths while exposing others' weaknesses.<n>Using a TrueSkill-based ranking system, we evaluate six LLMs and find that: (1) weaker models can reliably differentiate and score stronger ones, (2) LLM-based systems are capable of self-preferencing behavior, generating questions that align with their own capabilities, and (3) SKATE automatically surfaces fine
  arXiv  Detail & Related papers  (2025-08-08T08:16:40Z)
• Who is a Better Player: LLM against LLM [53.46608216197315]
  We propose an adversarial benchmarking framework to assess the comprehensive performance of Large Language Models (LLMs) through board games competition.<n>We introduce Qi Town, a specialized evaluation platform that supports 5 widely played games and involves 20 LLM-driven players.
  arXiv  Detail & Related papers  (2025-08-05T06:41:47Z)
• Benchmarking and Confidence Evaluation of LALMs For Temporal Reasoning [30.308743810639758]
  Large audio language models (LALMs) have to be evaluated on reasoning related tasks which are different from traditional classification or generation tasks.<n>We benchmark open-source LALMs and observe that they are consistently behind human capabilities on the tasks in the TREA dataset.<n>Our analysis shows that the accuracy and uncertainty metrics are not necessarily correlated and thus, points to a need for wholesome evaluation of LALMs for high-stakes applications.
  arXiv  Detail & Related papers  (2025-05-19T13:46:35Z)
• MoRE-LLM: Mixture of Rule Experts Guided by a Large Language Model [54.14155564592936]
  We propose a Mixture of Rule Experts guided by a Large Language Model (MoRE-LLM)<n>MoRE-LLM steers the discovery of local rule-based surrogates during training and their utilization for the classification task.<n>LLM is responsible for enhancing the domain knowledge alignment of the rules by correcting and contextualizing them.
  arXiv  Detail & Related papers  (2025-03-26T11:09:21Z)
• ZeroSumEval: An Extensible Framework For Scaling LLM Evaluation with Inter-Model Competition [14.753916893216129]
  We introduce ZeroSumEval, a dynamic, competition-based, and evolving evaluation framework for Large Language Models (LLMs)<n>ZeroSumEval encompasses a diverse suite of games, including security challenges (Capture the Flag), classic board games (chess), and knowledge tests (MathQuiz)
  arXiv  Detail & Related papers  (2025-03-10T16:54:27Z)
• Latent Factor Models Meets Instructions: Goal-conditioned Latent Factor Discovery without Task Supervision [50.45597801390757]
  Instruct-LF is a goal-oriented latent factor discovery system.<n>It integrates instruction-following ability with statistical models to handle noisy datasets.
  arXiv  Detail & Related papers  (2025-02-21T02:03:08Z)
• WiS Platform: Enhancing Evaluation of LLM-Based Multi-Agent Systems Through Game-Based Analysis [34.639887462203]
  We introduce an open, scalable, and real-time updated platform for accessing and analyzing the LLM-based MAS based on the games Who is Spy?" (WiS)<n>Our platform is featured with three main worths: (1) a unified model evaluate interface that supports models available on H Face; (2) real-time updated leaderboard for model evaluation; and (3) a comprehensive evaluation covering game-winning rates, attacking, defense strategies, and reasoning of LLMs.
  arXiv  Detail & Related papers  (2024-12-04T14:45:09Z)
• Evaluating and Advancing Multimodal Large Language Models in Perception Ability Lens [30.083110119139793]
  We introduce textbfAbilityLens, a unified benchmark designed to evaluate MLLMs in six key perception abilities.<n>We identify the strengths and weaknesses of current main-stream MLLMs, highlighting stability patterns and revealing a notable performance gap between state-of-the-art open-source and closed-source models.
  arXiv  Detail & Related papers  (2024-11-22T04:41:20Z)
• Understanding the Role of LLMs in Multimodal Evaluation Benchmarks [77.59035801244278]
  This paper investigates the role of the Large Language Model (LLM) backbone in Multimodal Large Language Models (MLLMs) evaluation. Our study encompasses four diverse MLLM benchmarks and eight state-of-the-art MLLMs. Key findings reveal that some benchmarks allow high performance even without visual inputs and up to 50% of error rates can be attributed to insufficient world knowledge in the LLM backbone.
  arXiv  Detail & Related papers  (2024-10-16T07:49:13Z)
• How Far Are We on the Decision-Making of LLMs? Evaluating LLMs' Gaming Ability in Multi-Agent Environments [83.78240828340681]
  GAMA($gamma$)-Bench is a new framework for evaluating Large Language Models' Gaming Ability in Multi-Agent environments.<n>$gamma$-Bench includes eight classical game theory scenarios and a dynamic scoring scheme specially designed to assess LLMs' performance.<n>Our results indicate GPT-3.5 demonstrates strong robustness but limited generalizability, which can be enhanced using methods like Chain-of-Thought.
  arXiv  Detail & Related papers  (2024-03-18T14:04:47Z)
• Leveraging Word Guessing Games to Assess the Intelligence of Large Language Models [105.39236338147715]
  The paper is inspired by the popular language game Who is Spy'' We develop DEEP to evaluate LLMs' expression and disguising abilities. We then introduce SpyGame, an interactive multi-agent framework.
  arXiv  Detail & Related papers  (2023-10-31T14:37:42Z)
• Improving Open Information Extraction with Large Language Models: A Study on Demonstration Uncertainty [52.72790059506241]
  Open Information Extraction (OIE) task aims at extracting structured facts from unstructured text. Despite the potential of large language models (LLMs) like ChatGPT as a general task solver, they lag behind state-of-the-art (supervised) methods in OIE tasks.
  arXiv  Detail & Related papers  (2023-09-07T01:35:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.