Full-Stage Pseudo Label Quality Enhancement for Weakly-supervised Temporal Action Localization

• URL: http://arxiv.org/abs/2407.08971v1
• Date: Fri, 12 Jul 2024 03:53:55 GMT
• Title: Full-Stage Pseudo Label Quality Enhancement for Weakly-supervised Temporal Action Localization
• Authors: Qianhan Feng, Wenshuo Li, Tong Lin, Xinghao Chen,
• Abstract summary: We propose a set of simple yet efficient pseudo label quality enhancement mechanisms to build our FuSTAL framework. FuSTAL achieves an average mAP of 50.8% on THUMOS'14, outperforming the previous best method by 1.2%, and becomes the first method to reach the milestone of 50%.
• Score: 11.010846827099936
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Weakly-supervised Temporal Action Localization (WSTAL) aims to localize actions in untrimmed videos using only video-level supervision. Latest WSTAL methods introduce pseudo label learning framework to bridge the gap between classification-based training and inferencing targets at localization, and achieve cutting-edge results. In these frameworks, a classification-based model is used to generate pseudo labels for a regression-based student model to learn from. However, the quality of pseudo labels in the framework, which is a key factor to the final result, is not carefully studied. In this paper, we propose a set of simple yet efficient pseudo label quality enhancement mechanisms to build our FuSTAL framework. FuSTAL enhances pseudo label quality at three stages: cross-video contrastive learning at proposal Generation-Stage, prior-based filtering at proposal Selection-Stage and EMA-based distillation at Training-Stage. These designs enhance pseudo label quality at different stages in the framework, and help produce more informative, less false and smoother action proposals. With the help of these comprehensive designs at all stages, FuSTAL achieves an average mAP of 50.8% on THUMOS'14, outperforming the previous best method by 1.2%, and becomes the first method to reach the milestone of 50%.

Related papers

• Bridge the Gap: From Weak to Full Supervision for Temporal Action Localization with PseudoFormer [13.153366072673915]
  We propose PseudoFormer, a novel framework that bridges the gap between weakly and fully-supervised WTAL. RickerFusion maps all predicted action proposals to a global shared space to generate pseudo labels with better quality. We leverage both snippet-level and proposal-level labels with different priors from the weak branch to train the regression-based model in the full branch. PseudoFormer achieves state-of-the-art WTAL results on the two commonly used benchmarks, THUMOS14 and ActivityNet1.3.
  arXiv  Detail & Related papers  (2025-04-21T05:00:07Z)
• BAL: Balancing Diversity and Novelty for Active Learning [53.289700543331925]
  We introduce a novel framework, Balancing Active Learning (BAL), which constructs adaptive sub-pools to balance diverse and uncertain data. Our approach outperforms all established active learning methods on widely recognized benchmarks by 1.20%.
  arXiv  Detail & Related papers  (2023-12-26T08:14:46Z)
• Semi-Supervised Class-Agnostic Motion Prediction with Pseudo Label Regeneration and BEVMix [59.55173022987071]
  We study the potential of semi-supervised learning for class-agnostic motion prediction. Our framework adopts a consistency-based self-training paradigm, enabling the model to learn from unlabeled data. Our method exhibits comparable performance to weakly and some fully supervised methods.
  arXiv  Detail & Related papers  (2023-12-13T09:32:50Z)
• One Class One Click: Quasi Scene-level Weakly Supervised Point Cloud Semantic Segmentation with Active Learning [29.493759008637532]
  We introduce One Class One Click (OCOC), a low cost yet informative quasi scene-level label, which encapsulates point-level and scene-level annotations. An active weakly supervised framework is proposed to leverage scarce labels by involving weak supervision from global and local perspectives. It considerably outperforms genuine scene-level weakly supervised methods by up to 25% in terms of average F1 score.
  arXiv  Detail & Related papers  (2022-11-23T01:23:26Z)
• Transductive CLIP with Class-Conditional Contrastive Learning [68.51078382124331]
  We propose Transductive CLIP, a novel framework for learning a classification network with noisy labels from scratch. A class-conditional contrastive learning mechanism is proposed to mitigate the reliance on pseudo labels. ensemble labels is adopted as a pseudo label updating strategy to stabilize the training of deep neural networks with noisy labels.
  arXiv  Detail & Related papers  (2022-06-13T14:04:57Z)
• PseCo: Pseudo Labeling and Consistency Training for Semi-Supervised Object Detection [42.75316070378037]
  We propose Noisy Pseudo box Learning (NPL) that includes Prediction-guided Label Assignment (PLA) and Positive-proposal Consistency Voting (PCV) On benchmark, our method, PSEudo labeling and COnsistency training (PseCo), outperforms the SOTA (Soft Teacher) by 2.0, 1.8, 2.0 points under 1%, 5%, and 10% labelling ratios.
  arXiv  Detail & Related papers  (2022-03-30T13:59:22Z)
• Active Self-Semi-Supervised Learning for Few Labeled Samples [4.713652957384158]
  Training deep models with limited annotations poses a significant challenge when applied to diverse practical domains. We propose a simple yet effective framework, active self-semi-supervised learning (AS3L) AS3L bootstraps semi-supervised models with prior pseudo-labels (PPL) We develop active learning and label propagation strategies to obtain accurate PPL.
  arXiv  Detail & Related papers  (2022-03-09T07:45:05Z)
• Label, Verify, Correct: A Simple Few Shot Object Detection Method [93.84801062680786]
  We introduce a simple pseudo-labelling method to source high-quality pseudo-annotations from a training set. We present two novel methods to improve the precision of the pseudo-labelling process. Our method achieves state-of-the-art or second-best performance compared to existing approaches.
  arXiv  Detail & Related papers  (2021-12-10T18:59:06Z)
• Novel Class Discovery in Semantic Segmentation [104.30729847367104]
  We introduce a new setting of Novel Class Discovery in Semantic (NCDSS) It aims at segmenting unlabeled images containing new classes given prior knowledge from a labeled set of disjoint classes. In NCDSS, we need to distinguish the objects and background, and to handle the existence of multiple classes within an image. We propose the Entropy-based Uncertainty Modeling and Self-training (EUMS) framework to overcome noisy pseudo-labels.
  arXiv  Detail & Related papers  (2021-12-03T13:31:59Z)
• Two-phase Pseudo Label Densification for Self-training based Domain Adaptation [93.03265290594278]
  We propose a novel Two-phase Pseudo Label Densification framework, referred to as TPLD. In the first phase, we use sliding window voting to propagate the confident predictions, utilizing intrinsic spatial-correlations in the images. In the second phase, we perform a confidence-based easy-hard classification. To ease the training process and avoid noisy predictions, we introduce the bootstrapping mechanism to the original self-training loss.
  arXiv  Detail & Related papers  (2020-12-09T02:35:25Z)
• Active and Incremental Learning with Weak Supervision [7.2288756536476635]
  In this work, we describe combinations of an incremental learning scheme and methods of active learning. An object detection task is evaluated in a continuous exploration context on the PASCAL VOC dataset. We also validate a weakly supervised system based on active and incremental learning in a real-world biodiversity application.
  arXiv  Detail & Related papers  (2020-01-20T13:21:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.