OmViD: Omni-supervised active learning for video action detection

• URL: http://arxiv.org/abs/2508.13983v1
• Date: Tue, 19 Aug 2025 16:19:58 GMT
• Title: OmViD: Omni-supervised active learning for video action detection
• Abstract summary: The study explores video-level tags, points, scribbles, bounding boxes, and pixel-level masks.<n>A novel 3D-superpixel approach is introduced to generate pseudo-labels from these annotations, enabling effective training.<n>The approach is validated on 101-24 and JHMDB-21 datasets.
• Score: 22.870129496984546
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Video action detection requires dense spatio-temporal annotations, which are both challenging and expensive to obtain. However, real-world videos often vary in difficulty and may not require the same level of annotation. This paper analyzes the appropriate annotation types for each sample and their impact on spatio-temporal video action detection. It focuses on two key aspects: 1) how to obtain varying levels of annotation for videos, and 2) how to learn action detection from different annotation types. The study explores video-level tags, points, scribbles, bounding boxes, and pixel-level masks. First, a simple active learning strategy is proposed to estimate the necessary annotation type for each video. Then, a novel spatio-temporal 3D-superpixel approach is introduced to generate pseudo-labels from these annotations, enabling effective training. The approach is validated on UCF101-24 and JHMDB-21 datasets, significantly cutting annotation costs with minimal performance loss.

Related papers

• Open-Vocabulary Spatio-Temporal Action Detection [59.91046192096296]
  Open-vocabulary-temporal action detection (OV-STAD) is an important fine-grained video understanding task. OV-STAD requires training a model on a limited set of base classes with box and label supervision. To better adapt the holistic VLM for the fine-grained action detection task, we carefully fine-tune it on the localized video region-text pairs.
  arXiv  Detail & Related papers  (2024-05-17T14:52:47Z)
• Semi-supervised Active Learning for Video Action Detection [8.110693267550346]
  We develop a novel semi-supervised active learning approach which utilizes both labeled as well as unlabeled data. We evaluate the proposed approach on three different benchmark datasets, UCF-24-101, JHMDB-21, and Youtube-VOS.
  arXiv  Detail & Related papers  (2023-12-12T11:13:17Z)
• FOCAL: A Cost-Aware Video Dataset for Active Learning [13.886774655927875]
  annotation-cost refers to the time it takes an annotator to label and quality-assure a given video sequence. We introduce a set of conformal active learning algorithms that take advantage of the sequential structure of video data. We show that the best conformal active learning method is cheaper than the best traditional active learning method by 113 hours.
  arXiv  Detail & Related papers  (2023-11-17T15:46:09Z)
• Towards Active Learning for Action Spotting in Association Football Videos [59.84375958757395]
  Analyzing football videos is challenging and requires identifying subtle and diverse-temporal patterns. Current algorithms face significant challenges when learning from limited annotated data. We propose an active learning framework that selects the most informative video samples to be annotated next.
  arXiv  Detail & Related papers  (2023-04-09T11:50:41Z)
• Weakly-Supervised Action Detection Guided by Audio Narration [50.4318060593995]
  We propose a model to learn from the narration supervision and utilize multimodal features, including RGB, motion flow, and ambient sound. Our experiments show that noisy audio narration suffices to learn a good action detection model, thus reducing annotation expenses.
  arXiv  Detail & Related papers  (2022-05-12T06:33:24Z)
• An Empirical Study of End-to-End Temporal Action Detection [82.64373812690127]
  Temporal action detection (TAD) is an important yet challenging task in video understanding. Rather than end-to-end learning, most existing methods adopt a head-only learning paradigm. We validate the advantage of end-to-end learning over head-only learning and observe up to 11% performance improvement.
  arXiv  Detail & Related papers  (2022-04-06T16:46:30Z)
• End-to-End Semi-Supervised Learning for Video Action Detection [23.042410033982193]
  We propose a simple end-to-end based approach effectively which utilizes the unlabeled data. Video action detection requires both, action class prediction as well as a-temporal consistency. We demonstrate the effectiveness of the proposed approach on two different action detection benchmark datasets.
  arXiv  Detail & Related papers  (2022-03-08T18:11:25Z)
• Few-Shot Action Localization without Knowing Boundaries [9.959844922120523]
  We show that it is possible to learn to localize actions in untrimmed videos when only one/few trimmed examples of the target action are available at test time. We propose a network that learns to estimate Temporal Similarity Matrices (TSMs) that model a fine-grained similarity pattern between pairs of videos. Our method achieves performance comparable or better to state-of-the-art fully-supervised, few-shot learning methods.
  arXiv  Detail & Related papers  (2021-06-08T07:32:43Z)
• RSPNet: Relative Speed Perception for Unsupervised Video Representation Learning [100.76672109782815]
  We study unsupervised video representation learning that seeks to learn both motion and appearance features from unlabeled video only. It is difficult to construct a suitable self-supervised task to well model both motion and appearance features. We propose a new way to perceive the playback speed and exploit the relative speed between two video clips as labels.
  arXiv  Detail & Related papers  (2020-10-27T16:42:50Z)
• Gabriella: An Online System for Real-Time Activity Detection in Untrimmed Security Videos [72.50607929306058]
  We propose a real-time online system to perform activity detection on untrimmed security videos. The proposed method consists of three stages: tubelet extraction, activity classification and online tubelet merging. We demonstrate the effectiveness of the proposed approach in terms of speed (100 fps) and performance with state-of-the-art results.
  arXiv  Detail & Related papers  (2020-04-23T22:20:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.