Unsupervised Temporal Action Localization via Self-paced Incremental Learning

• URL: http://arxiv.org/abs/2312.07384v1
• Date: Tue, 12 Dec 2023 16:00:55 GMT
• Title: Unsupervised Temporal Action Localization via Self-paced Incremental Learning
• Authors: Haoyu Tang, Han Jiang, Mingzhu Xu, Yupeng Hu, Jihua Zhu, Liqiang Nie
• Abstract summary: We present a novel self-paced incremental learning model to enhance clustering and localization training simultaneously. We design two (constant- and variable- speed) incremental instance learning strategies for easy-to-hard model training, thus ensuring the reliability of these video pseudolabels.
• Score: 57.55765505856969
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Recently, temporal action localization (TAL) has garnered significant interest in information retrieval community. However, existing supervised/weakly supervised methods are heavily dependent on extensive labeled temporal boundaries and action categories, which is labor-intensive and time-consuming. Although some unsupervised methods have utilized the ``iteratively clustering and localization'' paradigm for TAL, they still suffer from two pivotal impediments: 1) unsatisfactory video clustering confidence, and 2) unreliable video pseudolabels for model training. To address these limitations, we present a novel self-paced incremental learning model to enhance clustering and localization training simultaneously, thereby facilitating more effective unsupervised TAL. Concretely, we improve the clustering confidence through exploring the contextual feature-robust visual information. Thereafter, we design two (constant- and variable- speed) incremental instance learning strategies for easy-to-hard model training, thus ensuring the reliability of these video pseudolabels and further improving overall localization performance. Extensive experiments on two public datasets have substantiated the superiority of our model over several state-of-the-art competitors.

Related papers

• Learning Discriminative Spatio-temporal Representations for Semi-supervised Action Recognition [23.44320273156057]
  We propose an Adaptive Contrastive Learning(ACL) strategy and a Multi-scale Temporal Learning(MTL) strategy. ACL strategy assesses the confidence of all unlabeled samples by the class prototypes of the labeled data, and adaptively selects positive-negative samples from a pseudo-labeled sample bank to construct contrastive learning. MTL strategy could highlight informative semantics from long-term clips and integrate them into the short-term clip while suppressing noisy information.
  arXiv  Detail & Related papers  (2024-04-25T08:49:08Z)
• STAT: Towards Generalizable Temporal Action Localization [56.634561073746056]
  Weakly-supervised temporal action localization (WTAL) aims to recognize and localize action instances with only video-level labels. Existing methods suffer from severe performance degradation when transferring to different distributions. We propose GTAL, which focuses on improving the generalizability of action localization methods.
  arXiv  Detail & Related papers  (2024-04-20T07:56:21Z)
• TrACT: A Training Dynamics Aware Contrastive Learning Framework for Long-tail Trajectory Prediction [7.3292387742640415]
  We propose to incorporate richer training dynamics information into a prototypical contrastive learning framework. We conduct empirical evaluations of our approach using two large-scale naturalistic datasets.
  arXiv  Detail & Related papers  (2024-04-18T23:12:46Z)
• No More Shortcuts: Realizing the Potential of Temporal Self-Supervision [69.59938105887538]
  We propose a more challenging reformulation of temporal self-supervision as frame-level (rather than clip-level) recognition tasks. We demonstrate experimentally that our more challenging frame-level task formulations and the removal of shortcuts drastically improve the quality of features learned through temporal self-supervision.
  arXiv  Detail & Related papers  (2023-12-20T13:20:31Z)
• Weakly-Supervised Action Localization by Hierarchically-structured Latent Attention Modeling [19.683714649646603]
  Weakly-supervised action localization aims to recognize and localize action instancese in untrimmed videos with only video-level labels. Most existing models rely on multiple instance learning(MIL), where predictions of unlabeled instances are supervised by classifying labeled bags. We propose a novel attention-based hierarchically-structured latent model to learn the temporal variations of feature semantics.
  arXiv  Detail & Related papers  (2023-08-19T08:45:49Z)
• Weakly-Supervised Temporal Action Localization by Inferring Salient Snippet-Feature [26.7937345622207]
  Weakly-supervised temporal action localization aims to locate action regions and identify action categories in unsupervised videos simultaneously. Pseudo label generation is a promising strategy to solve the challenging problem, but the current methods ignore the natural temporal structure of the video. We propose a novel weakly-supervised temporal action localization method by inferring salient snippet-feature.
  arXiv  Detail & Related papers  (2023-03-22T06:08:34Z)
• Accelerating Self-Supervised Learning via Efficient Training Strategies [98.26556609110992]
  Time for training self-supervised deep networks remains an order of magnitude larger than its supervised counterparts. Motivated by these issues, this paper investigates reducing the training time of recent self-supervised methods.
  arXiv  Detail & Related papers  (2022-12-11T21:49:39Z)
• Active Learning with Effective Scoring Functions for Semi-Supervised Temporal Action Localization [15.031156121516211]
  This paper focuses on a rarely investigated yet practical task named semi-supervised TAL. We propose an effective active learning method, named AL-STAL. Experiment results show that AL-STAL outperforms the existing competitors and achieves satisfying performance compared with fully-supervised learning.
  arXiv  Detail & Related papers  (2022-08-31T13:39:38Z)
• Zero-Shot Temporal Action Detection via Vision-Language Prompting [134.26292288193298]
  We propose a novel zero-Shot Temporal Action detection model via Vision-LanguagE prompting (STALE) Our model significantly outperforms state-of-the-art alternatives. Our model also yields superior results on supervised TAD over recent strong competitors.
  arXiv  Detail & Related papers  (2022-07-17T13:59:46Z)
• Self-supervised Video Object Segmentation [76.83567326586162]
  The objective of this paper is self-supervised representation learning, with the goal of solving semi-supervised video object segmentation (a.k.a. dense tracking) We make the following contributions: (i) we propose to improve the existing self-supervised approach, with a simple, yet more effective memory mechanism for long-term correspondence matching; (ii) by augmenting the self-supervised approach with an online adaptation module, our method successfully alleviates tracker drifts caused by spatial-temporal discontinuity; (iv) we demonstrate state-of-the-art results among the self-supervised approaches on DAVIS-2017 and YouTube
  arXiv  Detail & Related papers  (2020-06-22T17:55:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.