Student Classroom Behavior Detection based on Spatio-Temporal Network and Multi-Model Fusion

• URL: http://arxiv.org/abs/2310.16267v4
• Date: Mon, 9 Sep 2024 10:52:54 GMT
• Title: Student Classroom Behavior Detection based on Spatio-Temporal Network and Multi-Model Fusion
• Authors: Fan Yang, Xiaofei Wang,
• Abstract summary: Deep learning methods to detect students' classroom behavior automatically is a promising approach for analyzing their class performance and improving teaching effectiveness. However, the lack of publicly available datasets on student behavior, as well as the high cost of manually labeling such datasets, pose significant challenges for researchers in this field. We proposed a method for extending the Student Classroom Scenarios dataset through image-temporal behavior datasets.
• Score: 6.069671582146248
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Using deep learning methods to detect students' classroom behavior automatically is a promising approach for analyzing their class performance and improving teaching effectiveness. However, the lack of publicly available spatio-temporal datasets on student behavior, as well as the high cost of manually labeling such datasets, pose significant challenges for researchers in this field. To address this issue, we proposed a method for extending the spatio-temporal behavior dataset in Student Classroom Scenarios (SCB-ST-Dataset4) through image dataset. Our SCB-ST-Dataset4 comprises 757265 images with 25810 labels, focusing on 3 behaviors: hand-raising, reading, writing. Our proposed method can rapidly generate spatio-temporal behavior datasets without requiring extra manual labeling. Furthermore, we proposed a Behavior Similarity Index (BSI) to explore the similarity of behaviors. We evaluated the dataset using the YOLOv5, YOLOv7, YOLOv8, and SlowFast algorithms, achieving a mean average precision (map) of up to 82.3%. Last, we fused multiple models to generate student behavior-related data from various perspectives. The experiment further demonstrates the effectiveness of our method. And SCB-ST-Dataset4 provides a robust foundation for future research in student behavior detection, potentially contributing to advancements in this field. The SCB-ST-Dataset4 is available for download at: https://github.com/Whiffe/SCB-dataset.

Related papers

• Adaptive Retention & Correction: Test-Time Training for Continual Learning [114.5656325514408]
  A common problem in continual learning is the classification layer's bias towards the most recent task. We name our approach Adaptive Retention & Correction (ARC) ARC achieves an average performance increase of 2.7% and 2.6% on the CIFAR-100 and Imagenet-R datasets.
  arXiv  Detail & Related papers  (2024-05-23T08:43:09Z)
• Incremental Self-training for Semi-supervised Learning [56.57057576885672]
  IST is simple yet effective and fits existing self-training-based semi-supervised learning methods. We verify the proposed IST on five datasets and two types of backbone, effectively improving the recognition accuracy and learning speed.
  arXiv  Detail & Related papers  (2024-04-14T05:02:00Z)
• SCB-Dataset3: A Benchmark for Detecting Student Classroom Behavior [6.878489963907169]
  Student Classroom Behavior dataset (SCB-dataset3) represents real-life scenarios. Our dataset comprises 5686 images with 45578 labels, focusing on six behaviors: hand-raising, reading, writing, using a phone, bowing the head, and leaning over the table.
  arXiv  Detail & Related papers  (2023-10-04T01:43:46Z)
• Student Classroom Behavior Detection based on Improved YOLOv7 [3.6119958671506707]
  We propose the Student Classroom Behavior Detection method, based on improved YOLOv7. First, we created the Student Classroom Behavior dataset (SCB-Dataset), which includes 18.4k labels and 4.2k images. To improve detection accuracy in crowded scenes, we integrated the biformer attention module and Wise-IoU into the YOLOv7 network. Experiments were conducted on the SCB-Dataset, and the model achieved an mAP@0.5 of 79%, resulting in a 1.8% improvement over previous results.
  arXiv  Detail & Related papers  (2023-06-06T00:01:40Z)
• Student Classroom Behavior Detection based on YOLOv7-BRA and Multi-Model Fusion [8.800332201027299]
  We propose the Student Classroom Behavior Detection system based on based on YOLOv7-BRA. We identified eight different behavior patterns, including standing, sitting, speaking, listening, walking, raising hands, reading, and writing. We conducted experiments on the SCB-Dataset, and YOLOv7-BRA achieved an mAP@0.5 of 87.1%, resulting in a 2.2% improvement over previous results.
  arXiv  Detail & Related papers  (2023-05-13T02:46:41Z)
• Hierarchical Supervision and Shuffle Data Augmentation for 3D Semi-Supervised Object Detection [90.32180043449263]
  State-of-the-art 3D object detectors are usually trained on large-scale datasets with high-quality 3D annotations. A natural remedy is to adopt semi-supervised learning (SSL) by leveraging a limited amount of labeled samples and abundant unlabeled samples. This paper introduces a novel approach of Hierarchical Supervision and Shuffle Data Augmentation (HSSDA), which is a simple yet effective teacher-student framework.
  arXiv  Detail & Related papers  (2023-04-04T02:09:32Z)
• DyG2Vec: Efficient Representation Learning for Dynamic Graphs [26.792732615703372]
  Temporal graph neural networks have shown promising results in learning inductive representations by automatically extracting temporal patterns. We present an efficient yet effective attention-based encoder that leverages temporal edge encodings and window-based subgraph sampling to generate task-agnostic embeddings.
  arXiv  Detail & Related papers  (2022-10-30T18:13:04Z)
• Is margin all you need? An extensive empirical study of active learning on tabular data [66.18464006872345]
  We analyze the performance of a variety of active learning algorithms on 69 real-world datasets from the OpenML-CC18 benchmark. Surprisingly, we find that the classical margin sampling technique matches or outperforms all others, including current state-of-art.
  arXiv  Detail & Related papers  (2022-10-07T21:18:24Z)
• Self-Supervised Human Activity Recognition with Localized Time-Frequency Contrastive Representation Learning [16.457778420360537]
  We propose a self-supervised learning solution for human activity recognition with smartphone accelerometer data. We develop a model that learns strong representations from accelerometer signals, while reducing the model's reliance on class labels. We evaluate the performance of the proposed solution on three datasets, namely MotionSense, HAPT, and HHAR.
  arXiv  Detail & Related papers  (2022-08-26T22:47:18Z)
• Learning from Temporal Spatial Cubism for Cross-Dataset Skeleton-based Action Recognition [88.34182299496074]
  Action labels are only available on a source dataset, but unavailable on a target dataset in the training stage. We utilize a self-supervision scheme to reduce the domain shift between two skeleton-based action datasets. By segmenting and permuting temporal segments or human body parts, we design two self-supervised learning classification tasks.
  arXiv  Detail & Related papers  (2022-07-17T07:05:39Z)
• Stance Detection Benchmark: How Robust Is Your Stance Detection? [65.91772010586605]
  Stance Detection (StD) aims to detect an author's stance towards a certain topic or claim. We introduce a StD benchmark that learns from ten StD datasets of various domains in a multi-dataset learning setting. Within this benchmark setup, we are able to present new state-of-the-art results on five of the datasets.
  arXiv  Detail & Related papers  (2020-01-06T13:37:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.