Sports Intelligence: Assessing the Sports Understanding Capabilities of Language Models through Question Answering from Text to Video

• URL: http://arxiv.org/abs/2406.14877v1
• Date: Fri, 21 Jun 2024 05:57:50 GMT
• Title: Sports Intelligence: Assessing the Sports Understanding Capabilities of Language Models through Question Answering from Text to Video
• Authors: Zhengbang Yang, Haotian Xia, Jingxi Li, Zezhi Chen, Zhuangdi Zhu, Weining Shen,
• Abstract summary: Reasoning over complex sports scenarios has posed significant challenges to current NLP technologies. Our evaluation spans from simple queries on basic rules and historical facts to complex, context-specific reasoning. We propose a new benchmark based on a comprehensive overview of existing sports datasets and provided extensive error analysis.
• Score: 5.885902974241053
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Understanding sports is crucial for the advancement of Natural Language Processing (NLP) due to its intricate and dynamic nature. Reasoning over complex sports scenarios has posed significant challenges to current NLP technologies which require advanced cognitive capabilities. Toward addressing the limitations of existing benchmarks on sports understanding in the NLP field, we extensively evaluated mainstream large language models for various sports tasks. Our evaluation spans from simple queries on basic rules and historical facts to complex, context-specific reasoning, leveraging strategies from zero-shot to few-shot learning, and chain-of-thought techniques. In addition to unimodal analysis, we further assessed the sports reasoning capabilities of mainstream video language models to bridge the gap in multimodal sports understanding benchmarking. Our findings highlighted the critical challenges of sports understanding for NLP. We proposed a new benchmark based on a comprehensive overview of existing sports datasets and provided extensive error analysis which we hope can help identify future research priorities in this field.

Related papers

• SPORTU: A Comprehensive Sports Understanding Benchmark for Multimodal Large Language Models [15.062299319625701]
  SPORTU is a benchmark designed to assess Multimodal Large Language Models (MLLMs) across multi-level sports reasoning tasks. SPORTU comprises two key components: SPORTU-text, featuring 900 multiple-choice questions with human-annotated explanations for rule comprehension and strategy understanding. SPORTU-video consists of 1,701 slow-motion video clips across 7 different sports and 12,048 QA pairs, designed to assess multi-level reasoning.
  arXiv  Detail & Related papers  (2024-10-11T02:58:38Z)
• OSL-ActionSpotting: A Unified Library for Action Spotting in Sports Videos [56.393522913188704]
  We introduce OSL-ActionSpotting, a Python library that unifies different action spotting algorithms to streamline research and applications in sports video analytics. We successfully integrated three cornerstone action spotting methods into OSL-ActionSpotting, achieving performance metrics that match those of the original, disparates.
  arXiv  Detail & Related papers  (2024-07-01T13:17:37Z)
• OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AI [73.75520820608232]
  We introduce OlympicArena, which includes 11,163 bilingual problems across both text-only and interleaved text-image modalities. These challenges encompass a wide range of disciplines spanning seven fields and 62 international Olympic competitions, rigorously examined for data leakage. Our evaluations reveal that even advanced models like GPT-4o only achieve a 39.97% overall accuracy, illustrating current AI limitations in complex reasoning and multimodal integration.
  arXiv  Detail & Related papers  (2024-06-18T16:20:53Z)
• SportQA: A Benchmark for Sports Understanding in Large Language Models [12.860652075730876]
  SportQA is a benchmark specifically designed for evaluating Large Language Models (LLMs) in the context of sports understanding. SportQA encompasses over 70,000 multiple-choice questions across three distinct difficulty levels. Our results reveal that while LLMs exhibit competent performance in basic sports knowledge, they struggle with more complex, scenario-based sports reasoning.
  arXiv  Detail & Related papers  (2024-02-24T17:12:10Z)
• Sports-QA: A Large-Scale Video Question Answering Benchmark for Complex and Professional Sports [90.79212954022218]
  We introduce the first dataset, named Sports-QA, specifically designed for the sports VideoQA task. Sports-QA dataset includes various types of questions, such as descriptions, chronologies, causalities, and counterfactual conditions. We propose a new Auto-Focus Transformer (AFT) capable of automatically focusing on particular scales of temporal information for question answering.
  arXiv  Detail & Related papers  (2024-01-03T02:22:34Z)
• A Survey of Deep Learning in Sports Applications: Perception, Comprehension, and Decision [41.427845300209945]
  Deep learning has the potential to revolutionize sports performance, with applications ranging from perception and comprehension to decision. This paper focuses on three main aspects: algorithms, datasets and virtual environments, and challenges.
  arXiv  Detail & Related papers  (2023-07-07T02:22:17Z)
• A Survey on Video Action Recognition in Sports: Datasets, Methods and Applications [60.3327085463545]
  We present a survey on video action recognition for sports analytics. We introduce more than ten types of sports, including team sports, such as football, basketball, volleyball, hockey and individual sports, such as figure skating, gymnastics, table tennis, diving and badminton. We develop a toolbox using PaddlePaddle, which supports football, basketball, table tennis and figure skating action recognition.
  arXiv  Detail & Related papers  (2022-06-02T13:19:36Z)
• Distantly Supervised Semantic Text Detection and Recognition for Broadcast Sports Videos Understanding [0.0]
  We study extremely accurate semantic text detection and recognition in sports clocks. We propose a novel distant supervision technique to automatically build sports clock datasets. We share our computational architecture pipeline to scale this system in industrial setting.
  arXiv  Detail & Related papers  (2021-10-31T23:59:29Z)
• Game Plan: What AI can do for Football, and What Football can do for AI [83.79507996785838]
  Predictive and prescriptive football analytics require new developments and progress at the intersection of statistical learning, game theory, and computer vision. We illustrate that football analytics is a game changer of tremendous value, in terms of not only changing the game of football itself, but also in terms of what this domain can mean for the field of AI.
  arXiv  Detail & Related papers  (2020-11-18T10:26:02Z)
• Interactive Fiction Game Playing as Multi-Paragraph Reading Comprehension with Reinforcement Learning [94.50608198582636]
  Interactive Fiction (IF) games with real human-written natural language texts provide a new natural evaluation for language understanding techniques. We take a novel perspective of IF game solving and re-formulate it as Multi-Passage Reading (MPRC) tasks.
  arXiv  Detail & Related papers  (2020-10-05T23:09:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.