Stabilising Learner Trajectories: A Doubly Robust Evaluation of AI-Guided Student Support using Activity Theory

• URL: http://arxiv.org/abs/2512.11154v1
• Date: Thu, 11 Dec 2025 22:28:12 GMT
• Title: Stabilising Learner Trajectories: A Doubly Robust Evaluation of AI-Guided Student Support using Activity Theory
• Authors: Teo Susnjak, Khalid Bakhshov, Anuradha Mathrani,
• Abstract summary: This study evaluates an AI-guided student support system at a large university using doubly robust score matching.<n>Results indicate that the intervention effectively stabilised precarious trajectories.<n>However, effects on the speed of qualification completion were positive but statistically constrained.
• Score: 1.2234742322758418
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While predictive models are increasingly common in higher education, causal evidence regarding the interventions they trigger remains rare. This study evaluates an AI-guided student support system at a large university using doubly robust propensity score matching. We advance the methodology for learning analytics evaluation by leveraging time-aligned, dynamic AI probability of success scores to match 1,859 treated students to controls, thereby mitigating the selection and immortal time biases often overlooked in observational studies. Results indicate that the intervention effectively stabilised precarious trajectories, and compared to the control group, supported students significantly reduced their course failure rates and achieved higher cumulative grades. However, effects on the speed of qualification completion were positive but statistically constrained. We interpreted these findings through Activity Theory, framing the intervention as a socio-technical brake that interrupts and slows the accumulation of academic failure among at-risk students. The student support-AI configuration successfully resolved the primary contradiction of immediate academic risk, but secondary contradictions within institutional structures limited the acceleration of degree completion. We conclude that while AI-enabled support effectively arrests decline, translating this stability into faster progression requires aligning intervention strategies with broader institutional governance.

Related papers

• Test-Time Learning of Causal Structure from Interventional Data [50.06913286558919]
  We propose TICL (Test-time Interventional Causal Learning), a novel method that synergizes Test-Time Training with Joint Causal Inference.<n>Specifically, we design a self-augmentation strategy to generate instance-specific training data at test time, effectively avoiding distribution shifts.<n>By integrating joint causal inference, we developed a PC-inspired two-phase supervised learning scheme, which effectively leverages self-augmented training data while ensuring theoretical identifiability.
  arXiv  Detail & Related papers  (2026-02-22T11:23:05Z)
• Transforming GenAI Policy to Prompting Instruction: An RCT of Scalable Prompting Interventions in a CS1 Course [8.222598094097867]
  We conducted a semester-long RCT with four ICAP framework-based instructional conditions varying in engagement intensity with a pre-test, immediate and delayed post-test and surveys.<n>We found that all conditions significantly improved prompting skills, with gains increasing progressively from Condition 1 to Condition 4.<n>For students with similar pre-test scores, higher learning gain in immediate post-test predict higher final exam score, though no direct between-group differences emerged.
  arXiv  Detail & Related papers  (2026-02-17T21:40:12Z)
• The Causal Effect of First-Time Academic Failure on University Dropout: Evidence from a Regression Discontinuity Design [0.0]
  This study estimates the causal effect of first-time academic failure on subsequent university attrition.<n> Contrary to conventional assumptions, the results indicate that marginal first-time failure is associated with a lower probability of subsequent dropout.
  arXiv  Detail & Related papers  (2026-01-09T18:08:15Z)
• Dynamics of Socio-Institutional Asynchrony in Generative AI: Analyzing the Relative Importance of Intervention Timing vs. Enforcement Efficiency via the Socio-Institutional Asynchrony Model (SIAM) [1.5906113067506233]
  Super-exponential growth of generative AI has intensified the institutional mismatch between the pace of technological diffusion and the speed of institutional adaptation.<n>This study proposes the Socio-Institutional Asynchrony Model, or SIAM, to quantitatively evaluate the relative effectiveness of two policy levers: intervention timing and enforcement efficiency.
  arXiv  Detail & Related papers  (2025-12-25T06:04:15Z)
• Designing AI-Resilient Assessments Using Interconnected Problems: A Theoretically Grounded and Empirically Validated Framework [0.0]
  The rapid adoption of generative AI has undermined traditional modular assessments in computing education.<n>This paper presents a theoretically grounded framework for designing AI-resilient assessments.
  arXiv  Detail & Related papers  (2025-12-11T15:53:19Z)
• UACER: An Uncertainty-Aware Critic Ensemble Framework for Robust Adversarial Reinforcement Learning [15.028168889991795]
  We propose a novel approach, Uncertainty-Aware Critic Ensemble for robust adversarial Reinforcement learning (UACER)<n>In this paper, we propose a novel approach, Uncertainty-Aware Critic Ensemble for robust adversarial Reinforcement learning (UACER)
  arXiv  Detail & Related papers  (2025-12-11T10:14:13Z)
• Causal Decomposition Analysis with Synergistic Interventions: A Triply-Robust Machine Learning Approach to Addressing Multiple Dimensions of Social Disparities [1.6586885757497203]
  We develop an extended causal decomposition analysis that simultaneously targets multiple causally ordered intervening factors.<n>We apply our method to a cohort of students from the High School Longitudinal Study.<n>We examine how two sequential interventions - equalizing the proportion of students who attend high-performing schools and equalizing enrollment in Algebra I by 9th grade across racial groups - may reduce these disparities.
  arXiv  Detail & Related papers  (2025-06-23T18:00:50Z)
• AI instructional agent improves student's perceived learner control and learning outcome: empirical evidence from a randomized controlled trial [29.45751702212421]
  This study examines the impact of an AI instructional agent on students' perceived learner control and academic performance in a medium demanding course with lecturing as the main teaching strategy.<n>Students in the AI instructional agent group reported significantly higher levels of perceived learner control compared to the other groups.
  arXiv  Detail & Related papers  (2025-05-28T16:13:27Z)
• Improving Domain Generalization in Self-supervised Monocular Depth Estimation via Stabilized Adversarial Training [61.35809887986553]
  We propose a general adversarial training framework, named Stabilized Conflict-optimization Adversarial Training (SCAT) SCAT integrates adversarial data augmentation into self-supervised MDE methods to achieve a balance between stability and generalization. Experiments on five benchmarks demonstrate that SCAT can achieve state-of-the-art performance and significantly improve the generalization capability of existing self-supervised MDE methods.
  arXiv  Detail & Related papers  (2024-11-04T15:06:57Z)
• The Pitfalls and Promise of Conformal Inference Under Adversarial Attacks [90.52808174102157]
  In safety-critical applications such as medical imaging and autonomous driving, it is imperative to maintain both high adversarial robustness to protect against potential adversarial attacks. A notable knowledge gap remains concerning the uncertainty inherent in adversarially trained models. This study investigates the uncertainty of deep learning models by examining the performance of conformal prediction (CP) in the context of standard adversarial attacks.
  arXiv  Detail & Related papers  (2024-05-14T18:05:19Z)
• Perturbation-Invariant Adversarial Training for Neural Ranking Models: Improving the Effectiveness-Robustness Trade-Off [107.35833747750446]
  adversarial examples can be crafted by adding imperceptible perturbations to legitimate documents. This vulnerability raises significant concerns about their reliability and hinders the widespread deployment of NRMs. In this study, we establish theoretical guarantees regarding the effectiveness-robustness trade-off in NRMs.
  arXiv  Detail & Related papers  (2023-12-16T05:38:39Z)
• Difficult Lessons on Social Prediction from Wisconsin Public Schools [32.90759447739759]
  Early warning systems assist in targeting interventions to individual students by predicting which students are at risk of dropping out. Despite significant investments in their widespread adoption, there remain large gaps in our understanding of the efficacy of EWS. We present empirical evidence that the prediction system accurately sorts students by their dropout risk. We find that it may have caused a single-digit percentage increase in graduation rates, though our empirical analyses cannot reliably rule out that there has been no positive treatment effect.
  arXiv  Detail & Related papers  (2023-04-13T00:59:12Z)
• Probabilistically Robust Learning: Balancing Average- and Worst-case Performance [105.87195436925722]
  We propose a framework called robustness probabilistic that bridges the gap between the accurate, yet brittle average case and the robust, yet conservative worst case. From a theoretical point of view, this framework overcomes the trade-offs between the performance and the sample-complexity of worst-case and average-case learning.
  arXiv  Detail & Related papers  (2022-02-02T17:01:38Z)
• Persistent Reinforcement Learning via Subgoal Curricula [114.83989499740193]
  Value-accelerated Persistent Reinforcement Learning (VaPRL) generates a curriculum of initial states. VaPRL reduces the interventions required by three orders of magnitude compared to episodic reinforcement learning.
  arXiv  Detail & Related papers  (2021-07-27T16:39:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.