Enlarging Instance-specific and Class-specific Information for Open-set Action Recognition

• URL: http://arxiv.org/abs/2303.15467v1
• Date: Sat, 25 Mar 2023 04:07:36 GMT
• Title: Enlarging Instance-specific and Class-specific Information for Open-set Action Recognition
• Authors: Jun Cen, Shiwei Zhang, Xiang Wang, Yixuan Pei, Zhiwu Qing, Yingya Zhang, Qifeng Chen
• Abstract summary: We find that features with richer semantic diversity can significantly improve the open-set performance under the same uncertainty scores. A novel Prototypical Similarity Learning (PSL) framework is proposed to keep the instance variance within the same class to retain more IS information.
• Score: 47.69171542776917
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Open-set action recognition is to reject unknown human action cases which are out of the distribution of the training set. Existing methods mainly focus on learning better uncertainty scores but dismiss the importance of feature representations. We find that features with richer semantic diversity can significantly improve the open-set performance under the same uncertainty scores. In this paper, we begin with analyzing the feature representation behavior in the open-set action recognition (OSAR) problem based on the information bottleneck (IB) theory, and propose to enlarge the instance-specific (IS) and class-specific (CS) information contained in the feature for better performance. To this end, a novel Prototypical Similarity Learning (PSL) framework is proposed to keep the instance variance within the same class to retain more IS information. Besides, we notice that unknown samples sharing similar appearances to known samples are easily misclassified as known classes. To alleviate this issue, video shuffling is further introduced in our PSL to learn distinct temporal information between original and shuffled samples, which we find enlarges the CS information. Extensive experiments demonstrate that the proposed PSL can significantly boost both the open-set and closed-set performance and achieves state-of-the-art results on multiple benchmarks. Code is available at https://github.com/Jun-CEN/PSL.

Related papers

• Views Can Be Deceiving: Improved SSL Through Feature Space Augmentation [27.609748213840138]
  In this work, we explore the impact of spurious features on Self-Supervised Learning (SSL) for visual representation learning. We show that commonly used augmentations in SSL can cause undesired invariances in the image space. We propose LateTVG to remove spurious information from these representations during pre-training, by regularizing later layers of the encoder via pruning.
  arXiv  Detail & Related papers  (2024-05-28T18:42:13Z)
• Learning Common Rationale to Improve Self-Supervised Representation for Fine-Grained Visual Recognition Problems [61.11799513362704]
  We propose learning an additional screening mechanism to identify discriminative clues commonly seen across instances and classes. We show that a common rationale detector can be learned by simply exploiting the GradCAM induced from the SSL objective.
  arXiv  Detail & Related papers  (2023-03-03T02:07:40Z)
• An Additive Instance-Wise Approach to Multi-class Model Interpretation [53.87578024052922]
  Interpretable machine learning offers insights into what factors drive a certain prediction of a black-box system. Existing methods mainly focus on selecting explanatory input features, which follow either locally additive or instance-wise approaches. This work exploits the strengths of both methods and proposes a global framework for learning local explanations simultaneously for multiple target classes.
  arXiv  Detail & Related papers  (2022-07-07T06:50:27Z)
• Weak Augmentation Guided Relational Self-Supervised Learning [80.0680103295137]
  We introduce a novel relational self-supervised learning (ReSSL) framework that learns representations by modeling the relationship between different instances. Our proposed method employs sharpened distribution of pairwise similarities among different instances as textitrelation metric. Experimental results show that our proposed ReSSL substantially outperforms the state-of-the-art methods across different network architectures.
  arXiv  Detail & Related papers  (2022-03-16T16:14:19Z)
• Active Learning for Open-set Annotation [38.739845944840454]
  We propose a new active learning framework called LfOSA, which boosts the classification performance with an effective sampling strategy to precisely detect examples from known classes for annotation. The experimental results show that the proposed method can significantly improve the selection quality of known classes, and achieve higher classification accuracy with lower annotation cost than state-of-the-art active learning methods.
  arXiv  Detail & Related papers  (2022-01-18T06:11:51Z)
• ReSSL: Relational Self-Supervised Learning with Weak Augmentation [68.47096022526927]
  Self-supervised learning has achieved great success in learning visual representations without data annotations. We introduce a novel relational SSL paradigm that learns representations by modeling the relationship between different instances. Our proposed ReSSL significantly outperforms the previous state-of-the-art algorithms in terms of both performance and training efficiency.
  arXiv  Detail & Related papers  (2021-07-20T06:53:07Z)
• ECKPN: Explicit Class Knowledge Propagation Network for Transductive Few-shot Learning [53.09923823663554]
  Class-level knowledge can be easily learned by humans from just a handful of samples. We propose an Explicit Class Knowledge Propagation Network (ECKPN) to address this problem. We conduct extensive experiments on four few-shot classification benchmarks, and the experimental results show that the proposed ECKPN significantly outperforms the state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-06-16T02:29:43Z)
• Self-supervised Detransformation Autoencoder for Representation Learning in Open Set Recognition [0.0]
  We propose a self-supervision method, Detransformation Autoencoder (DTAE) for the Open set recognition problem. Our proposed self-supervision method achieves significant gains in detecting the unknown class and classifying the known classes.
  arXiv  Detail & Related papers  (2021-05-28T02:45:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.