Improving Weakly Supervised Temporal Action Localization by Bridging Train-Test Gap in Pseudo Labels

• URL: http://arxiv.org/abs/2304.07978v1
• Date: Mon, 17 Apr 2023 03:47:41 GMT
• Title: Improving Weakly Supervised Temporal Action Localization by Bridging Train-Test Gap in Pseudo Labels
• Authors: Jingqiu Zhou, Linjiang Huang, Liang Wang, Si Liu, Hongsheng Li
• Abstract summary: Pseudo-label-based methods, which serve as an effective solution, have been widely studied recently. Existing methods generate pseudo labels during training and make predictions during testing under different pipelines or settings. We propose to generate high-quality pseudo labels from the predicted action boundaries.
• Score: 38.35756338815097
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The task of weakly supervised temporal action localization targets at generating temporal boundaries for actions of interest, meanwhile the action category should also be classified. Pseudo-label-based methods, which serve as an effective solution, have been widely studied recently. However, existing methods generate pseudo labels during training and make predictions during testing under different pipelines or settings, resulting in a gap between training and testing. In this paper, we propose to generate high-quality pseudo labels from the predicted action boundaries. Nevertheless, we note that existing post-processing, like NMS, would lead to information loss, which is insufficient to generate high-quality action boundaries. More importantly, transforming action boundaries into pseudo labels is quite challenging, since the predicted action instances are generally overlapped and have different confidence scores. Besides, the generated pseudo-labels can be fluctuating and inaccurate at the early stage of training. It might repeatedly strengthen the false predictions if there is no mechanism to conduct self-correction. To tackle these issues, we come up with an effective pipeline for learning better pseudo labels. Firstly, we propose a Gaussian weighted fusion module to preserve information of action instances and obtain high-quality action boundaries. Second, we formulate the pseudo-label generation as an optimization problem under the constraints in terms of the confidence scores of action instances. Finally, we introduce the idea of $\Delta$ pseudo labels, which enables the model with the ability of self-correction. Our method achieves superior performance to existing methods on two benchmarks, THUMOS14 and ActivityNet1.3, achieving gains of 1.9\% on THUMOS14 and 3.7\% on ActivityNet1.3 in terms of average mAP.

Related papers

• Towards Adaptive Pseudo-label Learning for Semi-Supervised Temporal Action Localization [10.233225586034665]
  Existing methods often filter pseudo labels based on strict conditions, leading to suboptimal pseudo-label ranking and selection. We propose a novel Adaptive Pseudo-label Learning framework to facilitate better pseudo-label selection. Our method achieves state-of-the-art performance under various semi-supervised settings.
  arXiv  Detail & Related papers  (2024-07-10T14:00:19Z)
• All Points Matter: Entropy-Regularized Distribution Alignment for Weakly-supervised 3D Segmentation [67.30502812804271]
  Pseudo-labels are widely employed in weakly supervised 3D segmentation tasks where only sparse ground-truth labels are available for learning. We propose a novel learning strategy to regularize the generated pseudo-labels and effectively narrow the gaps between pseudo-labels and model predictions.
  arXiv  Detail & Related papers  (2023-05-25T08:19:31Z)
• Exploiting Completeness and Uncertainty of Pseudo Labels for Weakly Supervised Video Anomaly Detection [149.23913018423022]
  Weakly supervised video anomaly detection aims to identify abnormal events in videos using only video-level labels. Two-stage self-training methods have achieved significant improvements by self-generating pseudo labels. We propose an enhancement framework by exploiting completeness and uncertainty properties for effective self-training.
  arXiv  Detail & Related papers  (2022-12-08T05:53:53Z)
• Label, Verify, Correct: A Simple Few Shot Object Detection Method [93.84801062680786]
  We introduce a simple pseudo-labelling method to source high-quality pseudo-annotations from a training set. We present two novel methods to improve the precision of the pseudo-labelling process. Our method achieves state-of-the-art or second-best performance compared to existing approaches.
  arXiv  Detail & Related papers  (2021-12-10T18:59:06Z)
• Learning Action Completeness from Points for Weakly-supervised Temporal Action Localization [15.603643098270409]
  We tackle the problem of localizing temporal intervals of actions with only a single frame label for each action instance for training. In this paper, we propose a novel framework, where dense pseudo-labels are generated to provide completeness guidance for the model.
  arXiv  Detail & Related papers  (2021-08-11T04:54:39Z)
• Rethinking Pseudo Labels for Semi-Supervised Object Detection [84.697097472401]
  We introduce certainty-aware pseudo labels tailored for object detection. We dynamically adjust the thresholds used to generate pseudo labels and reweight loss functions for each category to alleviate the class imbalance problem. Our approach improves supervised baselines by up to 10% AP using only 1-10% labeled data from COCO.
  arXiv  Detail & Related papers  (2021-06-01T01:32:03Z)
• Two-phase Pseudo Label Densification for Self-training based Domain Adaptation [93.03265290594278]
  We propose a novel Two-phase Pseudo Label Densification framework, referred to as TPLD. In the first phase, we use sliding window voting to propagate the confident predictions, utilizing intrinsic spatial-correlations in the images. In the second phase, we perform a confidence-based easy-hard classification. To ease the training process and avoid noisy predictions, we introduce the bootstrapping mechanism to the original self-training loss.
  arXiv  Detail & Related papers  (2020-12-09T02:35:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.