Starting Seatwork Earlier as a Valid Measure of Student Engagement

• URL: http://arxiv.org/abs/2505.13341v1
• Date: Mon, 19 May 2025 16:49:08 GMT
• Title: Starting Seatwork Earlier as a Valid Measure of Student Engagement
• Authors: Ashish Gurung, Jionghao Lin, Zhongtian Huang, Conrad Borchers, Ryan S. Baker, Vincent Aleven, Kenneth R. Koedinger,
• Abstract summary: We build on prior work on session-level measures to evaluate their reliability and predictive validity.<n>Our findings show that these measures exhibit moderate to high month-to-month reliability.<n>They enhance the prediction of final math scores beyond prior knowledge and gaming-the-system behaviors.
• Score: 2.364788972996476
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Prior work has developed a range of automated measures ("detectors") of student self-regulation and engagement from student log data. These measures have been successfully used to make discoveries about student learning. Here, we extend this line of research to an underexplored aspect of self-regulation: students' decisions about when to start and stop working on learning software during classwork. In the first of two analyses, we build on prior work on session-level measures (e.g., delayed start, early stop) to evaluate their reliability and predictive validity. We compute these measures from year-long log data from Cognitive Tutor for students in grades 8-12 (N = 222). Our findings show that these measures exhibit moderate to high month-to-month reliability (G > .75), comparable to or exceeding gaming-the-system behavior. Additionally, they enhance the prediction of final math scores beyond prior knowledge and gaming-the-system behaviors. The improvement in learning outcome predictions beyond time-on-task suggests they capture a broader motivational state tied to overall learning. The second analysis demonstrates the cross-system generalizability of these measures in i-Ready, where they predict state test scores for grade 7 students (N = 818). By leveraging log data, we introduce system-general naturally embedded measures that complement motivational surveys without extra instrumentation or disruption of instruction time. Our findings demonstrate the potential of session-level logs to mine valid and generalizable measures with broad applications in the predictive modeling of learning outcomes and analysis of learner self-regulation.

Related papers

• Reinforcement Learning via Implicit Imitation Guidance [49.88208134736617]
  A natural approach is to incorporate an imitation learning objective, either as regularization during training or to acquire a reference policy.<n>We propose to use prior data solely for guiding exploration via noise added to the policy, sidestepping the need for explicit behavior cloning constraints.<n>Our approach achieves up to 2-3x improvement over prior reinforcement learning from offline methods across seven simulated continuous control tasks.
  arXiv  Detail & Related papers  (2025-06-09T07:32:52Z)
• Existing Large Language Model Unlearning Evaluations Are Inconclusive [105.55899615056573]
  We show that some evaluations introduce substantial new information into the model, potentially masking true unlearning performance.<n>We demonstrate that evaluation outcomes vary significantly across tasks, undermining the generalizability of current evaluation routines.<n>We propose two principles for future unlearning evaluations: minimal information injection and downstream task awareness.
  arXiv  Detail & Related papers  (2025-05-31T19:43:00Z)
• Predicting Long-Term Student Outcomes from Short-Term EdTech Log Data [24.198449873743762]
  We investigate machine learning predictors using students' logs during their first few hours of usage.<n>Our findings suggest that short-term log usage data, from 2-5 hours, can be used to provide valuable signal about students' long-term external performance.
  arXiv  Detail & Related papers  (2024-12-20T01:05:23Z)
• A Survey of the Impact of Self-Supervised Pretraining for Diagnostic Tasks with Radiological Images [71.26717896083433]
  Self-supervised pretraining has been observed to be effective at improving feature representations for transfer learning. This review summarizes recent research into its usage in X-ray, computed tomography, magnetic resonance, and ultrasound imaging.
  arXiv  Detail & Related papers  (2023-09-05T19:45:09Z)
• A Predictive Model using Machine Learning Algorithm in Identifying Students Probability on Passing Semestral Course [0.0]
  This study employs classification for data mining techniques, and decision tree for algorithm. With the utilization of the newly discovered predictive model, the prediction of students probabilities to pass the current courses they take gives 0.7619 accuracy, 0.8333 precision, 0.8823 recall, and 0.8571 f1 score.
  arXiv  Detail & Related papers  (2023-04-12T01:57:08Z)
• ASPEST: Bridging the Gap Between Active Learning and Selective Prediction [56.001808843574395]
  Selective prediction aims to learn a reliable model that abstains from making predictions when uncertain. Active learning aims to lower the overall labeling effort, and hence human dependence, by querying the most informative examples. In this work, we introduce a new learning paradigm, active selective prediction, which aims to query more informative samples from the shifted target domain.
  arXiv  Detail & Related papers  (2023-04-07T23:51:07Z)
• What Should I Know? Using Meta-gradient Descent for Predictive Feature Discovery in a Single Stream of Experience [63.75363908696257]
  computational reinforcement learning seeks to construct an agent's perception of the world through predictions of future sensations. An open challenge in this line of work is determining from the infinitely many predictions that the agent could possibly make which predictions might best support decision-making. We introduce a meta-gradient descent process by which an agent learns what predictions to make, 2) the estimates for its chosen predictions, and 3) how to use those estimates to generate policies that maximize future reward.
  arXiv  Detail & Related papers  (2022-06-13T21:31:06Z)
• Early Performance Prediction using Interpretable Patterns in Programming Process Data [13.413990352918098]
  We leverage rich, fine-grained log data to build a model to predict student course outcomes. We evaluate our approach on a dataset from 106 students in a block-based, introductory programming course.
  arXiv  Detail & Related papers  (2021-02-10T22:46:45Z)
• Predicting MOOCs Dropout Using Only Two Easily Obtainable Features from the First Week's Activities [56.1344233010643]
  Several features are considered to contribute towards learner attrition or lack of interest, which may lead to disengagement or total dropout. This study aims to predict dropout early-on, from the first week, by comparing several machine-learning approaches.
  arXiv  Detail & Related papers  (2020-08-12T10:44:49Z)
• Counterfactual Predictions under Runtime Confounding [74.90756694584839]
  We study the counterfactual prediction task in the setting where all relevant factors are captured in the historical data. We propose a doubly-robust procedure for learning counterfactual prediction models in this setting.
  arXiv  Detail & Related papers  (2020-06-30T15:49:05Z)
• EPARS: Early Prediction of At-risk Students with Online and Offline Learning Behaviors [55.33024245762306]
  Early prediction of students at risk (STAR) is an effective and significant means to provide timely intervention for dropout and suicide. Existing works mostly rely on either online or offline learning behaviors which are not comprehensive enough to capture the whole learning processes. We propose a novel algorithm (EPARS) that could early predict STAR in a semester by modeling online and offline learning behaviors.
  arXiv  Detail & Related papers  (2020-06-06T12:56:26Z)
• Academic Performance Estimation with Attention-based Graph Convolutional Networks [17.985752744098267]
  Given a student's past data, the task of student's performance prediction is to predict a student's grades in future courses. Traditional methods for student's performance prediction usually neglect the underlying relationships between multiple courses. We propose a novel attention-based graph convolutional networks model for student's performance prediction.
  arXiv  Detail & Related papers  (2019-12-26T23:11:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.