Bag of States: A Non-sequential Approach to Video-based Engagement Measurement

• URL: http://arxiv.org/abs/2301.06730v1
• Date: Tue, 17 Jan 2023 07:12:34 GMT
• Title: Bag of States: A Non-sequential Approach to Video-based Engagement Measurement
• Authors: Ali Abedi, Chinchu Thomas, Dinesh Babu Jayagopi, and Shehroz S. Khan
• Abstract summary: Students' behavioral and emotional states need to be analyzed at fine-grained time scales in order to measure their level of engagement. Many existing approaches have developed sequential andtemporal models, such as recurrent neural networks, temporal convolutional networks, and three-dimensional convolutional neural networks, for measuring student engagement from videos. We develop bag-of-words-based models in which only occurrence of behavioral and emotional states of students is modeled and analyzed and not the order in which they occur.
• Score: 7.864500429933145
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automatic measurement of student engagement provides helpful information for instructors to meet learning program objectives and individualize program delivery. Students' behavioral and emotional states need to be analyzed at fine-grained time scales in order to measure their level of engagement. Many existing approaches have developed sequential and spatiotemporal models, such as recurrent neural networks, temporal convolutional networks, and three-dimensional convolutional neural networks, for measuring student engagement from videos. These models are trained to incorporate the order of behavioral and emotional states of students into video analysis and output their level of engagement. In this paper, backed by educational psychology, we question the necessity of modeling the order of behavioral and emotional states of students in measuring their engagement. We develop bag-of-words-based models in which only the occurrence of behavioral and emotional states of students is modeled and analyzed and not the order in which they occur. Behavioral and affective features are extracted from videos and analyzed by the proposed models to determine the level of engagement in an ordinal-output classification setting. Compared to the existing sequential and spatiotemporal approaches for engagement measurement, the proposed non-sequential approach improves the state-of-the-art results. According to experimental results, our method significantly improved engagement level classification accuracy on the IIITB Online SE dataset by 26% compared to sequential models and achieved engagement level classification accuracy as high as 66.58% on the DAiSEE student engagement dataset.

Related papers

• Engagement Measurement Based on Facial Landmarks and Spatial-Temporal Graph Convolutional Networks [2.4343669357792708]
  This paper introduces a novel, privacy-preserving method for engagement measurement from videos. It uses facial landmarks, which carry no personally identifiable information, extracted from videos via the MediaPipe deep learning solution. The proposed method is capable of being deployed on virtual learning platforms and measuring engagement in real-time.
  arXiv  Detail & Related papers  (2024-03-25T20:43:23Z)
• Skeleton2vec: A Self-supervised Learning Framework with Contextualized Target Representations for Skeleton Sequence [56.092059713922744]
  We show that using high-level contextualized features as prediction targets can achieve superior performance. Specifically, we propose Skeleton2vec, a simple and efficient self-supervised 3D action representation learning framework. Our proposed Skeleton2vec outperforms previous methods and achieves state-of-the-art results.
  arXiv  Detail & Related papers  (2024-01-01T12:08:35Z)
• A Control-Centric Benchmark for Video Prediction [69.22614362800692]
  We propose a benchmark for action-conditioned video prediction in the form of a control benchmark. Our benchmark includes simulated environments with 11 task categories and 310 task instance definitions. We then leverage our benchmark to study the effects of scaling model size, quantity of training data, and model ensembling.
  arXiv  Detail & Related papers  (2023-04-26T17:59:45Z)
• Multi-Layer Personalized Federated Learning for Mitigating Biases in Student Predictive Analytics [8.642174401125263]
  We propose a Multi-Layer Personalized Federated Learning (MLPFL) methodology to optimize inference accuracy over different layers of student grouping criteria. In our approach, personalized models for individual student subgroups are derived from a global model. Experiments on three real-world online course datasets show significant improvements achieved by our approach over existing student modeling benchmarks.
  arXiv  Detail & Related papers  (2022-12-05T17:27:28Z)
• Predicting student performance using sequence classification with time-based windows [1.5836913530330787]
  We show that accurate predictive models can be built based on sequential patterns derived from students' behavioral data. We present a methodology for capturing temporal aspects in behavioral data and analyze its influence on the predictive performance of the models. The results of our improved sequence classification technique are capable of predicting student performance with high levels of accuracy, reaching 90 percent for course-specific models.
  arXiv  Detail & Related papers  (2022-08-16T13:46:39Z)
• Mitigating Biases in Student Performance Prediction via Attention-Based Personalized Federated Learning [7.040747348755578]
  Traditional learning-based approaches to student modeling generalize poorly to underrepresented student groups due to biases in data availability. We propose a methodology for predicting student performance from their online learning activities that optimize inference accuracy over different demographic groups such as race and gender.
  arXiv  Detail & Related papers  (2022-08-02T00:22:20Z)
• Mixed Effects Neural ODE: A Variational Approximation for Analyzing the Dynamics of Panel Data [50.23363975709122]
  We propose a probabilistic model called ME-NODE to incorporate (fixed + random) mixed effects for analyzing panel data. We show that our model can be derived using smooth approximations of SDEs provided by the Wong-Zakai theorem. We then derive Evidence Based Lower Bounds for ME-NODE, and develop (efficient) training algorithms.
  arXiv  Detail & Related papers  (2022-02-18T22:41:51Z)
• Self-Regulated Learning for Egocentric Video Activity Anticipation [147.9783215348252]
  Self-Regulated Learning (SRL) aims to regulate the intermediate representation consecutively to produce representation that emphasizes the novel information in the frame of the current time-stamp. SRL sharply outperforms existing state-of-the-art in most cases on two egocentric video datasets and two third-person video datasets.
  arXiv  Detail & Related papers  (2021-11-23T03:29:18Z)
• Efficient Modelling Across Time of Human Actions and Interactions [92.39082696657874]
  We argue that current fixed-sized-temporal kernels in 3 convolutional neural networks (CNNDs) can be improved to better deal with temporal variations in the input. We study how we can better handle between classes of actions, by enhancing their feature differences over different layers of the architecture. The proposed approaches are evaluated on several benchmark action recognition datasets and show competitive results.
  arXiv  Detail & Related papers  (2021-10-05T15:39:11Z)
• Affect-driven Engagement Measurement from Videos [0.8545305424564517]
  We present a novel approach for video-based engagement measurement in virtual learning programs. Deep learning-based temporal, and traditional machine-learning-based non-temporal models are trained and validated. Our experiments show a state-of-the-art engagement level classification accuracy of 63.3% and correctly classifying disengagement videos.
  arXiv  Detail & Related papers  (2021-06-21T06:49:17Z)
• Unified Graph Structured Models for Video Understanding [93.72081456202672]
  We propose a message passing graph neural network that explicitly models relational-temporal relations. We show how our method is able to more effectively model relationships between relevant entities in the scene.
  arXiv  Detail & Related papers  (2021-03-29T14:37:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.