Few-Shot Continual Learning for Activity Recognition in Classroom Surveillance Images

• URL: http://arxiv.org/abs/2409.03354v1
• Date: Thu, 5 Sep 2024 08:55:56 GMT
• Title: Few-Shot Continual Learning for Activity Recognition in Classroom Surveillance Images
• Authors: Yilei Qian, Kanglei Geng, Kailong Chen, Shaoxu Cheng, Linfeng Xu, Hongliang Li, Fanman Meng, Qingbo Wu,
• Abstract summary: In real classroom settings, normal teaching activities account for a large proportion of samples, while rare non-teaching activities such as eating continue to appear. This requires a model that can learn non-teaching activities from few samples without forgetting the normal teaching activities.
• Score: 13.328067147864092
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The application of activity recognition in the "AI + Education" field is gaining increasing attention. However, current work mainly focuses on the recognition of activities in manually captured videos and a limited number of activity types, with little attention given to recognizing activities in surveillance images from real classrooms. In real classroom settings, normal teaching activities such as reading, account for a large proportion of samples, while rare non-teaching activities such as eating, continue to appear. This requires a model that can learn non-teaching activities from few samples without forgetting the normal teaching activities, which necessitates fewshot continual learning (FSCL) capability. To address this gap, we constructed a continual learning dataset focused on classroom surveillance image activity recognition called ARIC (Activity Recognition in Classroom). The dataset has advantages such as multiple perspectives, a wide variety of activities, and real-world scenarios, but it also presents challenges like similar activities and imbalanced sample distribution. To overcome these challenges, we designed a few-shot continual learning method that combines supervised contrastive learning (SCL) and an adaptive covariance classifier (ACC). During the base phase, we proposed a SCL approach based on feature augmentation to enhance the model's generalization ability. In the incremental phase, we employed an ACC to more accurately describe the distribution of new classes. Experimental results demonstrate that our method outperforms other existing methods on the ARIC dataset.

Related papers

• ARIC: An Activity Recognition Dataset in Classroom Surveillance Images [19.586321497367294]
  We construct a novel dataset focused on classroom surveillance image activity recognition called ARIC (Activity Recognition In Classroom) The ARIC dataset has advantages of multiple perspectives, 32 activity categories, three modalities, and real-world classroom scenarios. We hope that the ARIC dataset can act as a facilitator for future analysis and research for open teaching scenarios.
  arXiv  Detail & Related papers  (2024-10-16T07:59:07Z)
• Action-slot: Visual Action-centric Representations for Multi-label Atomic Activity Recognition in Traffic Scenes [23.284478293459856]
  Action-slot is a slot attention-based approach that learns visual action-centric representations. Our key idea is to design action slots that are capable of paying attention to regions where atomic activities occur. To address the limitation, we collect a synthetic dataset called TACO, which is four times larger than OATS.
  arXiv  Detail & Related papers  (2023-11-29T05:28:05Z)
• CUCL: Codebook for Unsupervised Continual Learning [129.91731617718781]
  The focus of this study is on Unsupervised Continual Learning (UCL), as it presents an alternative to Supervised Continual Learning. We propose a method named Codebook for Unsupervised Continual Learning (CUCL) which promotes the model to learn discriminative features to complete the class boundary. Our method significantly boosts the performances of supervised and unsupervised methods.
  arXiv  Detail & Related papers  (2023-11-25T03:08:50Z)
• Incremental Object Detection with CLIP [36.478530086163744]
  We propose a visual-language model such as CLIP to generate text feature embeddings for different class sets. We then employ super-classes to replace the unavailable novel classes in the early learning stage to simulate the incremental scenario. We incorporate the finely recognized detection boxes as pseudo-annotations into the training process, thereby further improving the detection performance.
  arXiv  Detail & Related papers  (2023-10-13T01:59:39Z)
• ALP: Action-Aware Embodied Learning for Perception [60.64801970249279]
  We introduce Action-Aware Embodied Learning for Perception (ALP) ALP incorporates action information into representation learning through a combination of optimizing a reinforcement learning policy and an inverse dynamics prediction objective. We show that ALP outperforms existing baselines in several downstream perception tasks.
  arXiv  Detail & Related papers  (2023-06-16T21:51:04Z)
• NTKCPL: Active Learning on Top of Self-Supervised Model by Estimating True Coverage [3.4806267677524896]
  We propose a novel active learning strategy, neural tangent kernel clustering-pseudo-labels (NTKCPL) It estimates empirical risk based on pseudo-labels and the model prediction with NTK approximation. We validate our method on five datasets, empirically demonstrating that it outperforms the baseline methods in most cases.
  arXiv  Detail & Related papers  (2023-06-07T01:43:47Z)
• Responsible Active Learning via Human-in-the-loop Peer Study [88.01358655203441]
  We propose a responsible active learning method, namely Peer Study Learning (PSL), to simultaneously preserve data privacy and improve model stability. We first introduce a human-in-the-loop teacher-student architecture to isolate unlabelled data from the task learner (teacher) on the cloud-side. During training, the task learner instructs the light-weight active learner which then provides feedback on the active sampling criterion.
  arXiv  Detail & Related papers  (2022-11-24T13:18:27Z)
• A Multi-label Continual Learning Framework to Scale Deep Learning Approaches for Packaging Equipment Monitoring [57.5099555438223]
  We study multi-label classification in the continual scenario for the first time. We propose an efficient approach that has a logarithmic complexity with regard to the number of tasks. We validate our approach on a real-world multi-label Forecasting problem from the packaging industry.
  arXiv  Detail & Related papers  (2022-08-08T15:58:39Z)
• Class-Incremental Learning for Action Recognition in Videos [44.923719189467164]
  We tackle catastrophic forgetting problem in the context of class-incremental learning for video recognition. Our framework addresses this challenging task by introducing time-channel importance maps and exploiting the importance maps for learning the representations of incoming examples. We evaluate the proposed approach on brand-new splits of class-incremental action recognition benchmarks constructed upon the UCF101, HMDB51, and Something-Something V2 datasets.
  arXiv  Detail & Related papers  (2022-03-25T12:15:49Z)
• Heterogeneous Contrastive Learning: Encoding Spatial Information for Compact Visual Representations [183.03278932562438]
  This paper presents an effective approach that adds spatial information to the encoding stage to alleviate the learning inconsistency between the contrastive objective and strong data augmentation operations. We show that our approach achieves higher efficiency in visual representations and thus delivers a key message to inspire the future research of self-supervised visual representation learning.
  arXiv  Detail & Related papers  (2020-11-19T16:26:25Z)
• ZSTAD: Zero-Shot Temporal Activity Detection [107.63759089583382]
  We propose a novel task setting called zero-shot temporal activity detection (ZSTAD), where activities that have never been seen in training can still be detected. We design an end-to-end deep network based on R-C3D as the architecture for this solution. Experiments on both the THUMOS14 and the Charades datasets show promising performance in terms of detecting unseen activities.
  arXiv  Detail & Related papers  (2020-03-12T02:40:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.