Language and Multimodal Models in Sports: A Survey of Datasets and Applications

• URL: http://arxiv.org/abs/2406.12252v1
• Date: Tue, 18 Jun 2024 03:59:26 GMT
• Title: Language and Multimodal Models in Sports: A Survey of Datasets and Applications
• Authors: Haotian Xia, Zhengbang Yang, Yun Zhao, Yuqing Wang, Jingxi Li, Rhys Tracy, Zhuangdi Zhu, Yuan-fang Wang, Hanjie Chen, Weining Shen,
• Abstract summary: Recent integration of Natural Language Processing (NLP) and multimodal models has advanced the field of sports analytics. This survey presents a comprehensive review of the datasets and applications driving these innovations post-2020.
• Score: 20.99857526324661
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Recent integration of Natural Language Processing (NLP) and multimodal models has advanced the field of sports analytics. This survey presents a comprehensive review of the datasets and applications driving these innovations post-2020. We overviewed and categorized datasets into three primary types: language-based, multimodal, and convertible datasets. Language-based and multimodal datasets are for tasks involving text or multimodality (e.g., text, video, audio), respectively. Convertible datasets, initially single-modal (video), can be enriched with additional annotations, such as explanations of actions and video descriptions, to become multimodal, offering future potential for richer and more diverse applications. Our study highlights the contributions of these datasets to various applications, from improving fan experiences to supporting tactical analysis and medical diagnostics. We also discuss the challenges and future directions in dataset development, emphasizing the need for diverse, high-quality data to support real-time processing and personalized user experiences. This survey provides a foundational resource for researchers and practitioners aiming to leverage NLP and multimodal models in sports, offering insights into current trends and future opportunities in the field.

Related papers

• Multimodal Large Language Models and Tunings: Vision, Language, Sensors, Audio, and Beyond [51.141270065306514]
  This tutorial aims to equip researchers, practitioners, and newcomers with the knowledge and skills to leverage multimodal AI. We will cover the latest multimodal datasets and pretrained models, including those beyond vision and language. Hands-on laboratories will offer practical experience with state-of-the-art multimodal models.
  arXiv  Detail & Related papers  (2024-10-08T01:41:56Z)
• MMSci: A Dataset for Graduate-Level Multi-Discipline Multimodal Scientific Understanding [59.41495657570397]
  This dataset includes figures such as schematic diagrams, simulated images, macroscopic/microscopic photos, and experimental visualizations. We developed benchmarks for scientific figure captioning and multiple-choice questions, evaluating six proprietary and over ten open-source models. The dataset and benchmarks will be released to support further research.
  arXiv  Detail & Related papers  (2024-07-06T00:40:53Z)
• HEMM: Holistic Evaluation of Multimodal Foundation Models [91.60364024897653]
  Multimodal foundation models can holistically process text alongside images, video, audio, and other sensory modalities. It is challenging to characterize and study progress in multimodal foundation models, given the range of possible modeling decisions, tasks, and domains.
  arXiv  Detail & Related papers  (2024-07-03T18:00:48Z)
• A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challenges [60.546677053091685]
  Large language models (LLMs) have unlocked novel opportunities for machine learning applications in the financial domain. We explore the application of LLMs on various financial tasks, focusing on their potential to transform traditional practices and drive innovation. We highlight this survey for categorizing the existing literature into key application areas, including linguistic tasks, sentiment analysis, financial time series, financial reasoning, agent-based modeling, and other applications.
  arXiv  Detail & Related papers  (2024-06-15T16:11:35Z)
• LLMs Meet Multimodal Generation and Editing: A Survey [89.76691959033323]
  This survey elaborates on multimodal generation and editing across various domains, comprising image, video, 3D, and audio. We summarize the notable advancements with milestone works in these fields and categorize these studies into LLM-based and CLIP/T5-based methods. We dig into tool-augmented multimodal agents that can leverage existing generative models for human-computer interaction.
  arXiv  Detail & Related papers  (2024-05-29T17:59:20Z)
• Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directions [11.786387517781328]
  Vision-Language Models (VLMs) are advanced models that can tackle more intricate tasks such as image captioning and visual question answering. Our classification organizes VLMs into three distinct categories: models dedicated to vision-language understanding, models that process multimodal inputs to generate unimodal (textual) outputs and models that both accept and produce multimodal inputs and outputs. We meticulously dissect each model, offering an extensive analysis of its foundational architecture, training data sources, as well as its strengths and limitations wherever possible.
  arXiv  Detail & Related papers  (2024-02-20T18:57:34Z)
• Multimodal Large Language Models: A Survey [36.06016060015404]
  Multimodal language models integrate multiple data types, such as images, text, language, audio, and other heterogeneity. This paper begins by defining the concept of multimodal and examining the historical development of multimodal algorithms. A practical guide is provided, offering insights into the technical aspects of multimodal models. Lastly, we explore the applications of multimodal models and discuss the challenges associated with their development.
  arXiv  Detail & Related papers  (2023-11-22T05:15:12Z)
• Multimodality Representation Learning: A Survey on Evolution, Pretraining and Its Applications [47.501121601856795]
  Multimodality Representation Learning is a technique of learning to embed information from different modalities and their correlations. Cross-modal interaction and complementary information from different modalities are crucial for advanced models to perform any multimodal task. This survey presents the literature on the evolution and enhancement of deep learning multimodal architectures.
  arXiv  Detail & Related papers  (2023-02-01T11:48:34Z)
• Multilingual Multimodality: A Taxonomical Survey of Datasets, Techniques, Challenges and Opportunities [10.721189858694396]
  We study the unification of multilingual and multimodal (MultiX) streams. We review the languages studied, gold or silver data with parallel annotations, and understand how these modalities and languages interact in modeling. We present an account of the modeling approaches along with their strengths and weaknesses to better understand what scenarios they can be used reliably.
  arXiv  Detail & Related papers  (2022-10-30T21:46:01Z)
• MELINDA: A Multimodal Dataset for Biomedical Experiment Method Classification [14.820951153262685]
  We introduce a new dataset, MELINDA, for Multimodal biomEdicaL experImeNt methoD clAssification. The dataset is collected in a fully automated distant supervision manner, where the labels are obtained from an existing curated database. We benchmark various state-of-the-art NLP and computer vision models, including unimodal models which only take either caption texts or images as inputs.
  arXiv  Detail & Related papers  (2020-12-16T19:11:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.