CoachMe: Decoding Sport Elements with a Reference-Based Coaching Instruction Generation Model

• URL: http://arxiv.org/abs/2509.11698v1
• Date: Mon, 15 Sep 2025 09:01:39 GMT
• Title: CoachMe: Decoding Sport Elements with a Reference-Based Coaching Instruction Generation Model
• Abstract summary: CoachMe is a reference-based model that analyzes the differences between a learner's motion and a reference under temporal and physical aspects.<n>We show how CoachMe adapts well to specific sports such as skating and boxing by learning from general movements and then leveraging limited data.
• Score: 14.887934352089742
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Motion instruction is a crucial task that helps athletes refine their technique by analyzing movements and providing corrective guidance. Although recent advances in multimodal models have improved motion understanding, generating precise and sport-specific instruction remains challenging due to the highly domain-specific nature of sports and the need for informative guidance. We propose CoachMe, a reference-based model that analyzes the differences between a learner's motion and a reference under temporal and physical aspects. This approach enables both domain-knowledge learning and the acquisition of a coach-like thinking process that identifies movement errors effectively and provides feedback to explain how to improve. In this paper, we illustrate how CoachMe adapts well to specific sports such as skating and boxing by learning from general movements and then leveraging limited data. Experiments show that CoachMe provides high-quality instructions instead of directions merely in the tone of a coach but without critical information. CoachMe outperforms GPT-4o by 31.6% in G-Eval on figure skating and by 58.3% on boxing. Analysis further confirms that it elaborates on errors and their corresponding improvement methods in the generated instructions. You can find CoachMe here: https://motionxperts.github.io/

Related papers

• Understanding Game Coaching on Gig Platforms [5.058358365779999]
  We conduct semi-structured interviews with 20 experienced freelance coaches across 17 competitive games on Fiverr.<n>We identify two structural conditions shaping this work: dual precarity, in which coaches navigate both gig platform instability and the lifecycle volatility of live-service games.<n>These coaches welcomed AI for administrative and analytic support but resisted its use in live interactions where trust, relational engagement, and situated judgment remained central.
  arXiv  Detail & Related papers  (2026-09-11T10:39:59Z)
• ExpertEdit: Learning Skill-Aware Motion Editing from Expert Videos [50.06747687804514]
  ExpertEdit is a framework for skill-driven motion editing trained exclusively on unpaired expert video demonstrations.<n>It produces localized skill improvements without paired supervision or manual edit guidance.<n>It outperforms state-of-the-art supervised motion editing methods on multiple metrics of motion realism and expert quality.
  arXiv  Detail & Related papers  (2026-04-12T05:25:33Z)
• SportSkills: Physical Skill Learning from Sports Instructional Videos [51.16409727318035]
  SportSkills is the first large-scale sports dataset geared towards physical skill learning with in-the-wild video.<n>We show that SportSkills unlocks the ability to understand fine-grained differences between physical actions.<n>We introduce the first large-scale task formulation of mistake-conditioned instructional video retrieval.
  arXiv  Detail & Related papers  (2026-03-26T08:29:35Z)
• CPMobius: Iterative Coach-Player Reasoning for Data-Free Reinforcement Learning [55.425576693143285]
  We introduce CPMbius (CPMobius), a collaborative Coach-Player paradigm for data-free reinforcement learning of reasoning models.<n>Unlike traditional adversarial self-play, CPMbius treats the Coach and Player as independent but cooperative roles.<n>CPMbius achieves substantial improvement without relying on any external training data, outperforming existing unsupervised approaches.
  arXiv  Detail & Related papers  (2026-02-03T01:38:53Z)
• Learning Skill-Attributes for Transferable Assessment in Video [56.813876909367856]
  Skill assessment from video entails rating the quality of a person's physical performance and explaining what could be done better.<n>Our CrossTrainer approach discovers skill-attributes, such as balance, control, and hand positioning.<n>By abstracting out the shared behaviors indicative of human skill, the proposed video representation generalizes substantially better than an array of existing techniques.
  arXiv  Detail & Related papers  (2025-11-17T23:53:06Z)
• ExpertAF: Expert Actionable Feedback from Video [81.46431188306397]
  We introduce a novel method to generate actionable feedback from video of a person doing a physical activity, such as basketball or soccer.<n>Our method takes a video demonstration and its accompanying 3D body pose and generates expert commentary describing what the person is doing well and what they could improve.<n>We show how to leverage Ego-Exo4D's [29] videos of skilled activity and expert commentary together with a strong language model to create a weakly-supervised training dataset for this task.
  arXiv  Detail & Related papers  (2024-08-01T16:13:07Z)
• AI coach for badminton [0.0]
  This study dissects video footage of badminton matches to extract insights into player kinetics and biomechanics. The research aims to derive predictive models that can suggest improvements in stance, technique, and muscle orientation. These recommendations are designed to mitigate erroneous techniques, reduce the risk of joint fatigue, and enhance overall performance.
  arXiv  Detail & Related papers  (2024-03-13T20:51:21Z)
• Effects of Multimodal Explanations for Autonomous Driving on Driving Performance, Cognitive Load, Expertise, Confidence, and Trust [2.9143343479274675]
  We tested the impact of an AI Coach's explanatory communications modeled after performance driving expert instructions. Results show AI coaching can effectively teach performance driving skills to novices. We suggest efficient, modality-appropriate explanations should be opted for when designing effective HMI communications.
  arXiv  Detail & Related papers  (2024-01-08T19:33:57Z)
• MAAIG: Motion Analysis And Instruction Generation [10.330867429217362]
  Many people engage in self-directed sports training at home but lack the real-time guidance of professional coaches. We propose a novel application framework called MAAIG. It can generate embedding vectors for each frame based on user-provided sports action videos.
  arXiv  Detail & Related papers  (2023-11-02T03:53:25Z)
• Understanding why shooters shoot -- An AI-powered engine for basketball performance profiling [70.54015529131325]
  Basketball is dictated by many variables, such as playstyle and game dynamics. It is crucial that the performance profiles can reflect the diverse playstyles. We present a tool that can visualize player performance profiles in a timely manner.
  arXiv  Detail & Related papers  (2023-03-17T01:13:18Z)
• A Survey on Video Action Recognition in Sports: Datasets, Methods and Applications [60.3327085463545]
  We present a survey on video action recognition for sports analytics. We introduce more than ten types of sports, including team sports, such as football, basketball, volleyball, hockey and individual sports, such as figure skating, gymnastics, table tennis, diving and badminton. We develop a toolbox using PaddlePaddle, which supports football, basketball, table tennis and figure skating action recognition.
  arXiv  Detail & Related papers  (2022-06-02T13:19:36Z)
• Coach2vec: autoencoding the playing style of soccer coaches [0.06091702876917279]
  Coach2vec is a workflow to capture the playing style of professional coaches using match event streams and artificial intelligence. It extracts ball possessions from each match, clusters them based on their similarity, and reconstructs the typical ball possessions of coaches. Experiments conducted on soccer-logs describing the last four seasons of the Italian first division, reveal interesting similarities between prominent coaches.
  arXiv  Detail & Related papers  (2021-06-29T14:32:38Z)
• Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Composition [88.26752130107259]
  In real-world multiagent systems, agents with different capabilities may join or leave without altering the team's overarching goals. We propose COPA, a coach-player framework to tackle this problem. We 1) adopt the attention mechanism for both the coach and the players; 2) propose a variational objective to regularize learning; and 3) design an adaptive communication method to let the coach decide when to communicate with the players.
  arXiv  Detail & Related papers  (2021-05-18T17:27:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.