Predicting Long-Term Student Outcomes from Short-Term EdTech Log Data

• URL: http://arxiv.org/abs/2412.15473v2
• Date: Wed, 15 Jan 2025 23:11:07 GMT
• Title: Predicting Long-Term Student Outcomes from Short-Term EdTech Log Data
• Authors: Ge Gao, Amelia Leon, Andrea Jetten, Jasmine Turner, Husni Almoubayyed, Stephen Fancsali, Emma Brunskill,
• Abstract summary: We investigate machine learning predictors using students' logs during their first few hours of usage.<n>Our findings suggest that short-term log usage data, from 2-5 hours, can be used to provide valuable signal about students' long-term external performance.
• Score: 24.198449873743762
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Educational stakeholders are often particularly interested in sparse, delayed student outcomes, like end-of-year statewide exams. The rare occurrence of such assessments makes it harder to identify students likely to fail such assessments, as well as making it slow for researchers and educators to be able to assess the effectiveness of particular educational tools. Prior work has primarily focused on using logs from students full usage (e.g. year-long) of an educational product to predict outcomes, or considered predictive accuracy using a few minutes to predict outcomes after a short (e.g. 1 hour) session. In contrast, we investigate machine learning predictors using students' logs during their first few hours of usage can provide useful predictive insight into those students' end-of-school year external assessment. We do this on three diverse datasets: from students in Uganda using a literacy game product, and from students in the US using two mathematics intelligent tutoring systems. We consider various measures of the accuracy of the resulting predictors, including its ability to identify students at different parts along the assessment performance distribution. Our findings suggest that short-term log usage data, from 2-5 hours, can be used to provide valuable signal about students' long-term external performance.

Related papers

• Starting Seatwork Earlier as a Valid Measure of Student Engagement [2.364788972996476]
  We build on prior work on session-level measures to evaluate their reliability and predictive validity.<n>Our findings show that these measures exhibit moderate to high month-to-month reliability.<n>They enhance the prediction of final math scores beyond prior knowledge and gaming-the-system behaviors.
  arXiv  Detail & Related papers  (2025-05-19T16:49:08Z)
• Detecting Unsuccessful Students in Cybersecurity Exercises in Two Different Learning Environments [0.37729165787434493]
  This paper develops automated tools to predict when a student is having difficulty. In a potential application, such models can aid instructors in detecting struggling students and providing targeted help.
  arXiv  Detail & Related papers  (2024-08-16T04:57:54Z)
• Explainable Few-shot Knowledge Tracing [48.877979333221326]
  We propose a cognition-guided framework that can track the student knowledge from a few student records while providing natural language explanations. Experimental results from three widely used datasets show that LLMs can perform comparable or superior to competitive deep knowledge tracing methods.
  arXiv  Detail & Related papers  (2024-05-23T10:07:21Z)
• Does Starting Deep Learning Homework Earlier Improve Grades? [63.20583929886827]
  Students who start a homework assignment earlier and spend more time on it should receive better grades on the assignment. Existing literature on the impact of time spent on homework is not clear-cut and comes mostly from K-12 education. We develop a hierarchical Bayesian model to help make principled conclusions about the impact on student success.
  arXiv  Detail & Related papers  (2023-09-30T09:34:30Z)
• A Predictive Model using Machine Learning Algorithm in Identifying Students Probability on Passing Semestral Course [0.0]
  This study employs classification for data mining techniques, and decision tree for algorithm. With the utilization of the newly discovered predictive model, the prediction of students probabilities to pass the current courses they take gives 0.7619 accuracy, 0.8333 precision, 0.8823 recall, and 0.8571 f1 score.
  arXiv  Detail & Related papers  (2023-04-12T01:57:08Z)
• ASPEST: Bridging the Gap Between Active Learning and Selective Prediction [56.001808843574395]
  Selective prediction aims to learn a reliable model that abstains from making predictions when uncertain. Active learning aims to lower the overall labeling effort, and hence human dependence, by querying the most informative examples. In this work, we introduce a new learning paradigm, active selective prediction, which aims to query more informative samples from the shifted target domain.
  arXiv  Detail & Related papers  (2023-04-07T23:51:07Z)
• Long-term Causal Inference Under Persistent Confounding via Data Combination [38.026740610259225]
  We study the identification and estimation of long-term treatment effects when both experimental and observational data are available. Since the long-term outcome is observed only after a long delay, it is not measured in the experimental data, but only recorded in the observational data.
  arXiv  Detail & Related papers  (2022-02-15T07:44:20Z)
• Explainable Student Performance Prediction With Personalized Attention for Explaining Why A Student Fails [0.5607676459156788]
  We propose a novel Explainable Student performance prediction method with Personalized Attention (ESPA) BiLSTM architecture extracts the semantic information in the paths with specific patterns. The ESPA consistently outperforms the other state-of-the-art models for student performance prediction.
  arXiv  Detail & Related papers  (2021-10-15T08:45:43Z)
• The application of predictive analytics to identify at-risk students in health professions education [0.0]
  Machine learning is used to predict which students are at risk of failing a national certifying exam. Predictions are made well in advance of the exam, such that educators can meaningfully intervene before students take the exam. The best predictive model has an accuracy of 93%, sensitivity of 69%, and specificity of 94%.
  arXiv  Detail & Related papers  (2021-08-05T03:55:53Z)
• Predicting student performance using data from an auto-grading system [0.0]
  We build decision-tree and linear-regression models with various features extracted from the Marmoset auto-grading system. We show that the linear-regression model using submission time intervals performs the best among all models in terms of Precision and F-Measure. We also show that for students who are misclassified into poor-performance students, they have the lowest actual grades in the linear-regression model among all models.
  arXiv  Detail & Related papers  (2021-02-02T03:02:39Z)
• Personalized Education in the AI Era: What to Expect Next? [76.37000521334585]
  The objective of personalized learning is to design an effective knowledge acquisition track that matches the learner's strengths and bypasses her weaknesses to meet her desired goal. In recent years, the boost of artificial intelligence (AI) and machine learning (ML) has unfolded novel perspectives to enhance personalized education.
  arXiv  Detail & Related papers  (2021-01-19T12:23:32Z)
• Predicting MOOCs Dropout Using Only Two Easily Obtainable Features from the First Week's Activities [56.1344233010643]
  Several features are considered to contribute towards learner attrition or lack of interest, which may lead to disengagement or total dropout. This study aims to predict dropout early-on, from the first week, by comparing several machine-learning approaches.
  arXiv  Detail & Related papers  (2020-08-12T10:44:49Z)
• EPARS: Early Prediction of At-risk Students with Online and Offline Learning Behaviors [55.33024245762306]
  Early prediction of students at risk (STAR) is an effective and significant means to provide timely intervention for dropout and suicide. Existing works mostly rely on either online or offline learning behaviors which are not comprehensive enough to capture the whole learning processes. We propose a novel algorithm (EPARS) that could early predict STAR in a semester by modeling online and offline learning behaviors.
  arXiv  Detail & Related papers  (2020-06-06T12:56:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.