Cross-Course Generalizability of SRL-Aligned Predictive Models Using Digital Learning Traces

• URL: http://arxiv.org/abs/2604.22812v1
• Date: Tue, 14 Apr 2026 15:30:13 GMT
• Title: Cross-Course Generalizability of SRL-Aligned Predictive Models Using Digital Learning Traces
• Abstract summary: STEM dropout rates remain high at universities, particularly in computer science programs with theory-intensive courses.<n>Digital learning environments now capture rich behavioral data that could help identify struggling students early.<n>This study analyzed multimodal digital-trace data from three undergraduate theoretical computer science courses at two universities.
• Score: 4.553564928877974
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: STEM dropout rates remain high at universities, particularly in computer science programs with theory-intensive courses. Digital learning environments now capture rich behavioral data that could help identify struggling students early, yet the generalizability of data-driven prediction models across courses and institutions remains uncertain. Guided by self-regulated learning (SRL) theory, this study analyzed multimodal digital-trace data from three undergraduate theoretical computer science courses (N1 = 137, N2 = 104, N3 = 148) at two universities. Weekly SRL-aligned digital-trace indicators were modeled using Elastic Net, Random Forest, and XGBoost to evaluate predictive performance over time and across settings, and model calibration both within and across courses. Early prediction of at-risk students was feasible, with SRL-related behaviors such as time management, effort regulation, and sustained engagement emerging as key predictors. While Random Forest achieved the highest in-sample accuracy, Elastic Net generalized more robustly across contexts. Out-of-sample accuracy and calibration declined between institutions with different base rates, underscoring the contextual nature of predictive analytics in higher education. These findings suggest that digital learning traces enable early identification of at-risk students within courses, but generalizing predictive models beyond their original context requires caution, particularly if the at-risk rates differ between contexts.

Related papers

• Jointly Predicting Courses and Grades Using a Transformer-Based Model [0.0]
  This paper introduces a TRansformer for Academic Course-grade Estimation (TRACE)<n>Our approach encodes courses on a per-semester basis to capture the effects for an upcoming semester.<n>We demonstrate that predicting courses taken in addition to the grades in those courses leads to significant improvements in prediction quality.
  arXiv  Detail & Related papers  (2026-08-13T16:05:22Z)
• Neural-Symbolic Knowledge Tracing: Injecting Educational Knowledge into Deep Learning for Responsible Learner Modelling [4.304713128122141]
  We propose Responsible-DKT, a neural-symbolic deep knowledge tracing approach.<n>It integrates symbolic educational knowledge (e.g., mastery and non-mastery rules) into sequential neural models for responsible learner modelling.<n>Experiments on a real-world dataset of students' math interactions show that Responsible-DKT outperforms both a neural-symbolic baseline and a fully data-driven PyTorch DKT model.
  arXiv  Detail & Related papers  (2026-04-09T13:49:07Z)
• On Neural Scaling Laws for Weather Emulation through Continual Training [76.70310203875866]
  We study neural scaling in Scientific Machine Learning, focusing on models for weather forecasting.<n>To analyze scaling behavior in as simple a setting as possible, we adopt a minimal, scalable, general-purpose Swin Transformer architecture.<n>We show that models trained in this minimalist way follow predictable scaling trends and even outperform standard cosine learning rate schedules.
  arXiv  Detail & Related papers  (2026-03-26T17:37:25Z)
• How and Why LLMs Generalize: A Fine-Grained Analysis of LLM Reasoning from Cognitive Behaviors to Low-Level Patterns [51.02752099869218]
  Large Language Models (LLMs) display strikingly different generalization behaviors.<n>We introduce a novel benchmark that decomposes reasoning into atomic core skills.<n>We show that RL-tuned models maintain more stable behavioral profiles and resist collapse in reasoning skills, whereas SFT models exhibit sharper drift and overfit to surface patterns.
  arXiv  Detail & Related papers  (2025-12-30T08:16:20Z)
• Entropy-Guided Token Dropout: Training Autoregressive Language Models with Limited Domain Data [89.96277093034547]
  We introduce EntroDrop, an entropy-guided token dropout method that functions as structured data regularization.<n>We show that EntroDrop consistently outperforms standard regularization baselines and maintains robust performance throughout extended multi-epoch training.
  arXiv  Detail & Related papers  (2025-12-29T12:35:51Z)
• Ranking-Based At-Risk Student Prediction Using Federated Learning and Differential Features [4.21051987964486]
  This study proposes a method that combines federated learning and differential features to address privacy concerns.<n>To evaluate the proposed method, a model for predicting at-risk students was trained using data from 1,136 students across 12 courses conducted over 4 years.<n>The trained models were also applicable for early prediction, achieving high performance in detecting at-risk students in earlier stages of the semester.
  arXiv  Detail & Related papers  (2025-05-14T11:12:30Z)
• Global Convergence of Continual Learning on Non-IID Data [51.99584235667152]
  We provide a general and comprehensive theoretical analysis for continual learning of regression models.<n>We establish the almost sure convergence results of continual learning under a general data condition for the first time.
  arXiv  Detail & Related papers  (2025-03-24T10:06:07Z)
• In-Context Linear Regression Demystified: Training Dynamics and Mechanistic Interpretability of Multi-Head Softmax Attention [52.159541540613915]
  We study how multi-head softmax attention models are trained to perform in-context learning on linear data.<n>Our results reveal that in-context learning ability emerges from the trained transformer as an aggregated effect of its architecture and the underlying data distribution.
  arXiv  Detail & Related papers  (2025-03-17T02:00:49Z)
• Modeling Behavior Change for Multi-model At-Risk Students Early Prediction (extended version) [10.413751893289056]
  Current models primarily identify students with consistently poor performance through simple and discrete behavioural patterns.<n>We have developed an innovative prediction model, Multimodal- ChangePoint Detection (MCPD), utilizing the textual teacher remark data and numerical grade data from middle schools.<n>Our model achieves an accuracy range of 70- 75%, with an average outperforming baseline algorithms by approximately 5-10%.
  arXiv  Detail & Related papers  (2025-02-19T11:16:46Z)
• Accurate Multi-Category Student Performance Forecasting at Early Stages of Online Education Using Neural Networks [2.195766695109612]
  This study introduces a novel neural network-based approach capable of accurately predicting student performance.<n>The proposed model predicts outcomes in Distinction, Fail, Pass, and Withdrawn categories.<n>The results indicate that the prediction accuracy of the proposed method is about 25% more than the existing state-of-the-art.
  arXiv  Detail & Related papers  (2024-12-08T13:37:30Z)
• SimSCOOD: Systematic Analysis of Out-of-Distribution Generalization in Fine-tuned Source Code Models [58.78043959556283]
  We study the behaviors of models under different fine-tuning methodologies, including full fine-tuning and Low-Rank Adaptation (LoRA) fine-tuning methods. Our analysis uncovers that LoRA fine-tuning consistently exhibits significantly better OOD generalization performance than full fine-tuning across various scenarios.
  arXiv  Detail & Related papers  (2022-10-10T16:07:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.