Shot2Tactic-Caption: Multi-Scale Captioning of Badminton Videos for Tactical Understanding

• URL: http://arxiv.org/abs/2510.14617v1
• Date: Thu, 16 Oct 2025 12:24:51 GMT
• Title: Shot2Tactic-Caption: Multi-Scale Captioning of Badminton Videos for Tactical Understanding
• Abstract summary: We propose bfShot2Tactic-Caption, a novel framework for semantic and temporal multi-scale video captioning in badminton.<n>Shot2Tactic-Caption adopts a dual-branch design, with both branches including a visual encoder, atemporal Transformer encoder, and a Transformer-based decoder.<n>For tactic captioning, we further incorporate a shot-wise prompt-guided mechanism, where the predicted tactic type state are embedded as prompts and injected into the decoder.
• Score: 10.526240385265256
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Tactical understanding in badminton involves interpreting not only individual actions but also how tactics are dynamically executed over time. In this paper, we propose \textbf{Shot2Tactic-Caption}, a novel framework for semantic and temporal multi-scale video captioning in badminton, capable of generating shot-level captions that describe individual actions and tactic-level captions that capture how these actions unfold over time within a tactical execution. We also introduce the Shot2Tactic-Caption Dataset, the first badminton captioning dataset containing 5,494 shot captions and 544 tactic captions. Shot2Tactic-Caption adopts a dual-branch design, with both branches including a visual encoder, a spatio-temporal Transformer encoder, and a Transformer-based decoder to generate shot and tactic captions. To support tactic captioning, we additionally introduce a Tactic Unit Detector that identifies valid tactic units, tactic types, and tactic states (e.g., Interrupt, Resume). For tactic captioning, we further incorporate a shot-wise prompt-guided mechanism, where the predicted tactic type and state are embedded as prompts and injected into the decoder via cross-attention. The shot-wise prompt-guided mechanism enables our system not only to describe successfully executed tactics but also to capture tactical executions that are temporarily interrupted and later resumed. Experimental results demonstrate the effectiveness of our framework in generating both shot and tactic captions. Ablation studies show that the ResNet50-based spatio-temporal encoder outperforms other variants, and that shot-wise prompt structuring leads to more coherent and accurate tactic captioning.

Related papers

• BFMD: A Full-Match Badminton Dense Dataset for Dense Shot Captioning [10.526240385265256]
  We introduce the first Badminton Full Match Dense dataset, with 19 broadcast matches covering over 20 hours of play, each annotated with a shot caption.<n>The dataset provides hierarchical annotations including match segments, rally events, and dense rally-level multimodal annotations such as shot types, shuttle trajectories, player pose keypoints, and shot captions.<n>We develop a VideoMAE-based multimodal captioning framework with a Semantic Feedback mechanism that leverages shot semantics to guide caption generation and improve semantic consistency.
  arXiv  Detail & Related papers  (2026-03-26T15:09:12Z)
• Imagine How To Change: Explicit Procedure Modeling for Change Captioning [18.938737195128347]
  Existing methods operate on static image pairs, ignoring the rich temporal dynamics of the change procedure.<n>We introduce ProCap, a novel framework that reformulates change modeling from static image comparison to dynamic procedure modeling.
  arXiv  Detail & Related papers  (2026-03-06T07:05:39Z)
• SGCap: Decoding Semantic Group for Zero-shot Video Captioning [14.484825416367338]
  Zero-shot video captioning aims to generate sentences for describing videos without training the model on video-text pairs.<n>We propose a Semantic Group Captioning (SGCap) method for zero-shot video captioning.
  arXiv  Detail & Related papers  (2025-08-02T09:05:45Z)
• Progress-Aware Video Frame Captioning [55.23366888264651]
  We propose ProgressCaptioner, a captioning model designed to capture the fine-grained temporal dynamics within an action sequence.<n>We develop the FrameCap dataset to support training and the FrameCapEval benchmark to assess caption quality.<n>Results demonstrate that ProgressCaptioner significantly surpasses leading captioning models.
  arXiv  Detail & Related papers  (2024-12-03T01:21:28Z)
• Learning text-to-video retrieval from image captioning [59.81537951811595]
  We describe a protocol to study text-to-video retrieval training with unlabeled videos. We assume (i) no access to labels for any videos, and (ii) access to labeled images in the form of text. We show that automatically labeling video frames with image captioning allows text-to-video retrieval training.
  arXiv  Detail & Related papers  (2024-04-26T15:56:08Z)
• I See Dead People: Gray-Box Adversarial Attack on Image-To-Text Models [0.0]
  We present a gray-box adversarial attack on image-to-text, both untargeted and targeted. Our attack operates in a gray-box manner, requiring no knowledge about the decoder module. We also show that our attacks fool the popular open-source platform Hugging Face.
  arXiv  Detail & Related papers  (2023-06-13T07:35:28Z)
• DeCap: Decoding CLIP Latents for Zero-Shot Captioning via Text-Only Training [73.74291217502928]
  We propose a simple framework, named DeCap, for zero-shot captioning. We introduce a lightweight visual-aware language decoder. We project the visual embedding into the CLIP text embedding space, while the projected embedding retains the information of the visual input.
  arXiv  Detail & Related papers  (2023-03-06T11:02:47Z)
• Prefix Conditioning Unifies Language and Label Supervision [84.11127588805138]
  We show that dataset biases negatively affect pre-training by reducing the generalizability of learned representations. In experiments, we show that this simple technique improves the performance in zero-shot image recognition accuracy and robustness to the image-level distribution shift.
  arXiv  Detail & Related papers  (2022-06-02T16:12:26Z)
• Controllable Video Captioning with an Exemplar Sentence [89.78812365216983]
  We propose a novel Syntax Modulated Caption Generator (SMCG) incorporated in an encoder-decoder-reconstructor architecture. SMCG takes video semantic representation as an input, and conditionally modulates the gates and cells of long short-term memory network. We conduct experiments by collecting auxiliary exemplar sentences for two public video captioning datasets.
  arXiv  Detail & Related papers  (2021-12-02T09:24:45Z)
• Optimizing Latency for Online Video CaptioningUsing Audio-Visual Transformers [54.705393237822044]
  This paper proposes a novel approach to optimize each caption's output timing based on a trade-off between latency and caption quality. An audio-visual Trans-former is trained to generate ground-truth captions using only a small portion of all video frames. A CNN-based timing detector is also trained to detect a proper output timing, where the captions generated by the two Trans-formers become sufficiently close to each other.
  arXiv  Detail & Related papers  (2021-08-04T16:20:00Z)
• Controlled Caption Generation for Images Through Adversarial Attacks [85.66266989600572]
  We study adversarial examples for vision and language models, which typically adopt a Convolutional Neural Network (i.e., CNN) for image feature extraction and a Recurrent Neural Network (RNN) for caption generation. In particular, we investigate attacks on the visual encoder's hidden layer that is fed to the subsequent recurrent network. We propose a GAN-based algorithm for crafting adversarial examples for neural image captioning that mimics the internal representation of the CNN.
  arXiv  Detail & Related papers  (2021-07-07T07:22:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.