Boundary-Centric Active Learning for Temporal Action Segmentation

• URL: http://arxiv.org/abs/2604.15173v1
• Date: Thu, 16 Apr 2026 15:50:21 GMT
• Title: Boundary-Centric Active Learning for Temporal Action Segmentation
• Abstract summary: We introduce B-ACT, a clip-budgeted active learning framework that explicitly allocates supervision to high-leverage boundary regions.<n>B-ACT operates in a hierarchical two-stage loop: (i) it ranks and queries unlabeled videos using predictive uncertainty, and (ii) within each selected video, it selects the top-$K$ boundaries.
• Score: 0.34785805439031336
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Temporal action segmentation (TAS) demands dense temporal supervision, yet most of the annotation cost in untrimmed videos is spent identifying and refining action transitions, where segmentation errors concentrate and small temporal shifts disproportionately degrade segmental metrics. We introduce B-ACT, a clip-budgeted active learning framework that explicitly allocates supervision to these high-leverage boundary regions. B-ACT operates in a hierarchical two-stage loop: (i) it ranks and queries unlabeled videos using predictive uncertainty, and (ii) within each selected video, it detects candidate transitions from the current model predictions and selects the top-$K$ boundaries via a novel boundary score that fuses neighborhood uncertainty, class ambiguity, and temporal predictive dynamics. Importantly, our annotation protocol requests labels for only the boundary frames while still training on boundary-centered clips to exploit temporal context through the model's receptive field. Extensive experiments on GTEA, 50Salads, and Breakfast demonstrate that boundary-centric supervision delivers strong label efficiency and consistently surpasses representative TAS active learning baselines and prior state of the art under sparse budgets, with the largest gains on datasets where boundary placement dominates edit and overlap-based F1 scores.

Related papers

• Boundary Voting Network for Ambiguity-Aware Timestamp-Supervised Action Segmentation [45.84397725976746]
  Timestamp-supervised action segmentation aims to segment and classify actions in untrimmed videos with a random frame annotated per action.<n>We introduce the boundary voting network that mitigates feature ambiguity by hierarchically propagating video-level global prior knowledge into local action-transiting regions.
  arXiv  Detail & Related papers  (2026-09-08T02:58:54Z)
• Temporal Action Localization with Enhanced Instant Discriminability [66.76095239972094]
  Temporal action detection (TAD) aims to detect all action boundaries and their corresponding categories in an untrimmed video. We propose a one-stage framework named TriDet to resolve imprecise predictions of action boundaries by existing methods. Experimental results demonstrate the robustness of TriDet and its state-of-the-art performance on multiple TAD datasets.
  arXiv  Detail & Related papers  (2023-09-11T16:17:50Z)
• Video Activity Localisation with Uncertainties in Temporal Boundary [74.7263952414899]
  Methods for video activity localisation over time assume implicitly that activity temporal boundaries are determined and precise. In unscripted natural videos, different activities transit smoothly, so that it is intrinsically ambiguous to determine in labelling precisely when an activity starts and ends over time. We introduce Elastic Moment Bounding (EMB) to accommodate flexible and adaptive activity temporal boundaries.
  arXiv  Detail & Related papers  (2022-06-26T16:45:56Z)
• Temporal Transductive Inference for Few-Shot Video Object Segmentation [27.140141181513425]
  Few-shot object segmentation (FS-VOS) aims at segmenting video frames using a few labelled examples of classes not seen during initial training. Key to our approach is the use of both global and local temporal constraints. Empirically, our model outperforms state-of-the-art meta-learning approaches in terms of mean intersection over union on YouTube-VIS by 2.8%.
  arXiv  Detail & Related papers  (2022-03-27T14:08:30Z)
• Transferable Knowledge-Based Multi-Granularity Aggregation Network for Temporal Action Localization: Submission to ActivityNet Challenge 2021 [33.840281113206444]
  This report presents an overview of our solution used in the submission to 2021 HACS Temporal Action localization Challenge. We use Temporal Context Aggregation Network (TCANet) to generate high-quality action proposals. We also adopt an additional module to transfer the knowledge from trimmed videos to untrimmed videos. Our proposed scheme achieves 39.91 and 29.78 average mAP on the challenge testing set of supervised and weakly-supervised temporal action localization track respectively.
  arXiv  Detail & Related papers  (2021-07-27T06:18:21Z)
• Temporal Context Aggregation Network for Temporal Action Proposal Refinement [93.03730692520999]
  Temporal action proposal generation is a challenging yet important task in the video understanding field. Current methods still suffer from inaccurate temporal boundaries and inferior confidence used for retrieval. We propose TCANet to generate high-quality action proposals through "local and global" temporal context aggregation.
  arXiv  Detail & Related papers  (2021-03-24T12:34:49Z)
• Boundary-sensitive Pre-training for Temporal Localization in Videos [124.40788524169668]
  We investigate model pre-training for temporal localization by introducing a novel boundary-sensitive pretext ( BSP) task. With the synthesized boundaries, BSP can be simply conducted via classifying the boundary types. Extensive experiments show that the proposed BSP is superior and complementary to the existing action classification based pre-training counterpart.
  arXiv  Detail & Related papers  (2020-11-21T17:46:24Z)
• Reinforcement Learning for Weakly Supervised Temporal Grounding of Natural Language in Untrimmed Videos [134.78406021194985]
  We focus on the weakly supervised setting of this task that merely accesses to coarse video-level language description annotation without temporal boundary. We propose a emphBoundary Adaptive Refinement (BAR) framework that resorts to reinforcement learning to guide the process of progressively refining the temporal boundary.
  arXiv  Detail & Related papers  (2020-09-18T03:32:47Z)
• MS-TCN++: Multi-Stage Temporal Convolutional Network for Action Segmentation [87.16030562892537]
  We propose a multi-stage architecture for the temporal action segmentation task. The first stage generates an initial prediction that is refined by the next ones. Our models achieve state-of-the-art results on three datasets.
  arXiv  Detail & Related papers  (2020-06-16T14:50:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.