Causal Modelling of Support Interventions for Student Competency Assessment

• URL: http://arxiv.org/abs/2608.24632v1
• Date: Tue, 25 Aug 2026 14:46:00 GMT
• Title: Causal Modelling of Support Interventions for Student Competency Assessment
• Abstract summary: We advocate adopting a structural causal modelling approach to educational assessment.<n>We analyse the practical relevance of forms of reasoning that remain inaccessible to standard associative models.<n>Although our protocol requires the structural equations to be elicited from experts, the necessary information is purely logical.
• Score: 42.46717736329823
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Accurate assessment of student competencies is essential for enabling educators to identify individual needs, design targeted interventions, and evaluate the effectiveness of educational strategies. Empirical assessment procedures are typically grounded in psychometric models, such as item response theory, which relate student competence levels to performance on assessment tasks. In this paper, we advocate adopting a structural causal modelling approach to educational assessment, moving beyond probabilistic belief updating toward a framework that explicitly supports interventional and counterfactual reasoning. We propose a corresponding protocol for its construction and analyse the practical relevance of forms of reasoning that remain inaccessible to standard associative models, including the explicit modelling of interventions such as hints and the related counterfactual scenario analysis. Although our protocol requires the structural equations to be elicited from experts, the necessary information is purely logical and does not rely on probabilistic, less tenable assumptions. We illustrate the approach using data from an assessment that employs complex tasks designed to measure compulsory school student algorithmic skills.

Related papers

• From Token Probabilities to Semantic Constraints: Towards Declarative Probabilistic Evaluation of Language Models [6.885801584985341]
  We propose a declarative probabilistic framework for pre-training evaluation that makes the semantic structure of model behavior explicit.<n>ModelLog specifies evaluation targets as symbolic constraints over token-level predictions and measures how strongly a model's distribution satisfies those constraints.<n>We show that these evaluation scores can also be interpreted as losses, whose gradients reflect logical strength, informativeness, and variable-level sensitivity.
  arXiv  Detail & Related papers  (2026-09-11T20:42:23Z)
• Practitioner Beliefs and Behaviors in AI-Enhanced Education: DOT Framework Survey Evidence [0.0]
  The study investigates AI familiarity, usage patterns, design-oriented practices, and pedagogical beliefs.<n>Results indicate that practitioners hold favorable views of AI as a pedagogical support while maintaining strong commitments to human oversight and critical evaluation.
  arXiv  Detail & Related papers  (2026-05-27T19:42:31Z)
• Design Principles and Observable Indicators for AI-Enabled Pedagogical Accompaniment: Evidence from the Amico Dual-Mode Prototype in Italy and China [0.0]
  This paper proposes a framework for AI-enabled pedagogical accompaniment, grounded in a human-in-command approach.<n>Building on the Amico project, we operationalize the concept of a relational bridge as a sequence of micro-mediations.<n>The paper outlines an initial cross-context exploration of the prototype in Italy and China and discusses how the two interaction modes, AmicoMio and AmicoTuo, can be used.
  arXiv  Detail & Related papers  (2026-05-20T03:32:07Z)
• Pitfalls in Evaluating Interpretability Agents [91.49742416116635]
  We build an agentic system in which a research agent iteratively designs experiments and refines hypotheses.<n>Our work demonstrates fundamental challenges in evaluating complex automated interpretability systems.
  arXiv  Detail & Related papers  (2026-03-20T16:27:17Z)
• Measurement to Meaning: A Validity-Centered Framework for AI Evaluation [12.55408229639344]
  We provide a structured approach for reasoning about the types of evaluative claims that can be made given the available evidence.<n>Our framework is well-suited for the contemporary paradigm in machine learning.
  arXiv  Detail & Related papers  (2025-05-13T20:36:22Z)
• Causality can systematically address the monsters under the bench(marks) [64.36592889550431]
  Benchmarks are plagued by various biases, artifacts, or leakage.<n>Models may behave unreliably due to poorly explored failure modes.<n> causality offers an ideal framework to systematically address these challenges.
  arXiv  Detail & Related papers  (2025-02-07T17:01:37Z)
• STRICTA: Structured Reasoning in Critical Text Assessment for Peer Review and Beyond [68.47402386668846]
  We introduce Structured Reasoning In Critical Text Assessment (STRICTA) to model text assessment as an explicit, step-wise reasoning process.<n>STRICTA breaks down the assessment into a graph of interconnected reasoning steps drawing on causality theory.<n>We apply STRICTA to a dataset of over 4000 reasoning steps from roughly 40 biomedical experts on more than 20 papers.
  arXiv  Detail & Related papers  (2024-09-09T06:55:37Z)
• Towards a Unified Framework for Evaluating Explanations [0.6138671548064356]
  We argue that explanations serve as mediators between models and stakeholders, whether for intrinsically interpretable models or opaque black-box models. We illustrate these criteria, as well as specific evaluation methods, using examples from an ongoing study of an interpretable neural network for predicting a particular learner behavior.
  arXiv  Detail & Related papers  (2024-05-22T21:49:28Z)
• Provable Representation with Efficient Planning for Partial Observable Reinforcement Learning [74.67655210734338]
  In most real-world reinforcement learning applications, state information is only partially observable, which breaks the Markov decision process assumption. We develop a representation-based perspective that leads to a coherent framework and tractable algorithmic approach for practical reinforcement learning from partial observations. We empirically demonstrate the proposed algorithm can surpass state-of-the-art performance with partial observations across various benchmarks.
  arXiv  Detail & Related papers  (2023-11-20T23:56:58Z)
• Modelling Assessment Rubrics through Bayesian Networks: a Pragmatic Approach [40.06500618820166]
  This paper presents an approach to deriving a learner model directly from an assessment rubric. We illustrate how the approach can be applied to automatize the human assessment of an activity developed for testing computational thinking skills.
  arXiv  Detail & Related papers  (2022-09-07T10:09:12Z)
• Evaluations and Methods for Explanation through Robustness Analysis [117.7235152610957]
  We establish a novel set of evaluation criteria for such feature based explanations by analysis. We obtain new explanations that are loosely necessary and sufficient for a prediction. We extend the explanation to extract the set of features that would move the current prediction to a target class.
  arXiv  Detail & Related papers  (2020-05-31T05:52:05Z)
• Instance-Based Learning of Span Representations: A Case Study through Named Entity Recognition [48.06319154279427]
  We present a method of instance-based learning that learns similarities between spans. Our method enables to build models that have high interpretability without sacrificing performance.
  arXiv  Detail & Related papers  (2020-04-29T23:32:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.