Interpreting Learning Under Competing Models: Joint and Stepwise Approaches for Dynamic Cognitive Diagnosis

• URL: http://arxiv.org/abs/2606.06804v1
• Date: Fri, 05 Jun 2026 01:08:48 GMT
• Title: Interpreting Learning Under Competing Models: Joint and Stepwise Approaches for Dynamic Cognitive Diagnosis
• Abstract summary: We analyse data from two reading games measuring vocabulary and comprehension from Grade 2 to Grade 3.<n>We show that this decision can change substantive conclusions about how learners develop.<n>A simulation study identifies when the two analyses diverge and shows that joint analysis is more reliable when the item-skill structure is uncertain.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Digital learning environments record learners' responses to individual items, making it possible to study the development of specific skills rather than overall scores. Drawing conclusions about learning from these data requires a model that links responses to latent skills and tracks how mastery changes over time. When the skills measured by each item are unknown, the analyst must decide whether to estimate this structure, the Q-matrix, jointly with the learning process, or to establish it first and study learning afterwards. We show that this decision can change substantive conclusions about how learners develop. Using dynamic cognitive diagnostic models, we analyse data from two reading games measuring vocabulary and comprehension from Grade 2 to Grade 3, with item-text embeddings providing prior information for the unknown Q-matrix. A joint analysis and a bias-corrected stepwise analysis agree that most learners move toward mastering both skills, but disagree about how many remain only partially proficient at Grade 3, changing how reading progress would be reported. A simulation study identifies when the two analyses diverge and shows that joint analysis is more reliable when the item-skill structure is uncertain and the item pool changes between grades. We provide R code for both analyses.

Related papers

• Through the WordStream Glass: Revisiting Quantitative Encoding for Qualitative Learning Analytics [0.6524460254566904]
  Data-driven learning analytics can surface trends across a student cohort over time, helping instructors improve the learning environment.<n>WordStream, a visualization idiom for topic evolution, has been instantiated in two platforms toward this goal: the Journal Data Dashboard, for analyzing formative assessments, and WordStream Maker, for authoring custom visualizations.<n>Here we examine the reverse direction: what can qualitative education research tell us about building better visualization tools?
  arXiv  Detail & Related papers  (2026-06-17T05:14:09Z)
• Towards Knowledgeable Deep Research: Framework and Benchmark [105.87257445617922]
  This paper introduces Knowledgeable Deep Research (KDR), which requires DR agents to generate reports with both structured and unstructured knowledge.<n>We propose the Hybrid Knowledge Analysis framework (HKA), a multi-agent architecture that integrates texts, figures, and tables into coherent multimodal reports.<n> Experimental results demonstrate that HKA consistently outperforms most existing DR agents on general-purpose and knowledge-centric metrics.
  arXiv  Detail & Related papers  (2026-04-09T02:06:27Z)
• Benchmarking Large Language Models for Personalized Guidance in AI-Enhanced Learning [4.990353320509215]
  Large Language Models (LLMs) are increasingly envisioned as intelligent assistants for personalized learning.<n>This study presents an empirical comparison of three state-of-the-art LLMs on a tutoring task simulating a realistic learning setting.
  arXiv  Detail & Related papers  (2025-09-02T14:21:59Z)
• Unveiling the Learning Mind of Language Models: A Cognitive Framework and Empirical Study [45.82081693725339]
  Large language models (LLMs) have shown impressive capabilities across tasks such as mathematics, coding, and reasoning.<n>Yet their learning ability, which is crucial for adapting to dynamic environments and acquiring new knowledge, remains underexplored.
  arXiv  Detail & Related papers  (2025-06-16T13:24:50Z)
• KnowCoder-V2: Deep Knowledge Analysis [64.63893361811968]
  We propose a textbfKnowledgeable textbfDeep textbfResearch (textbfKDR) framework that empowers deep research with deep knowledge analysis capability.<n>It introduces an independent knowledge organization phase to preprocess large-scale, domain-relevant data into systematic knowledge offline.<n>It then extends deep research with an additional kind of reasoning steps that perform complex knowledge computation in an online manner.
  arXiv  Detail & Related papers  (2025-06-07T18:01:25Z)
• When Models Know More Than They Can Explain: Quantifying Knowledge Transfer in Human-AI Collaboration [79.69935257008467]
  We introduce Knowledge Integration and Transfer Evaluation (KITE), a conceptual and experimental framework for Human-AI knowledge transfer capabilities.<n>We conduct the first large-scale human study (N=118) explicitly designed to measure it.<n>In our two-phase setup, humans first ideate with an AI on problem-solving strategies, then independently implement solutions, isolating model explanations' influence on human understanding.
  arXiv  Detail & Related papers  (2025-06-05T20:48:16Z)
• Learning Beyond Pattern Matching? Assaying Mathematical Understanding in LLMs [58.09253149867228]
  This paper assesses the domain knowledge of LLMs through its understanding of different mathematical skills required to solve problems. Motivated by the use of LLMs as a general scientific assistant, we propose textitNTKEval to assess changes in LLM's probability distribution. Our systematic analysis finds evidence of domain understanding during in-context learning. Certain instruction-tuning leads to similar performance changes irrespective of training on different data, suggesting a lack of domain understanding across different skills.
  arXiv  Detail & Related papers  (2024-05-24T12:04:54Z)
• A Mathematical Theory for Learning Semantic Languages by Abstract Learners [9.139188656944429]
  We develop a mathematical theory to explain the emergence of learned skills, taking the learning process into account. We demonstrate the emergence of learned skills when the ratio of the number of training texts to the number of skills exceeds a certain threshold. We use site percolation analysis to derive the conditions for the existence of a giant component in the skill association graph.
  arXiv  Detail & Related papers  (2024-04-10T13:50:46Z)
• Meta-Cognitive Analysis: Evaluating Declarative and Procedural Knowledge in Datasets and Large Language Models [47.33702059464214]
  Declarative knowledge and procedural knowledge are two key parts in meta-cognitive theory. This paper provides ground-truth knowledge for LLMs and evaluating the effective score.
  arXiv  Detail & Related papers  (2024-03-14T05:34:35Z)
• KIWI: A Dataset of Knowledge-Intensive Writing Instructions for Answering Research Questions [63.307317584926146]
  Large language models (LLMs) adapted to follow user instructions are now widely deployed as conversational agents. In this work, we examine one increasingly common instruction-following task: providing writing assistance to compose a long-form answer. We construct KIWI, a dataset of knowledge-intensive writing instructions in the scientific domain.
  arXiv  Detail & Related papers  (2024-03-06T17:16:44Z)
• A Literature Review of Literature Reviews in Pattern Analysis and Machine Intelligence [51.26815896167173]
  We present a comprehensive tertiary analysis of PAMI reviews along three complementary dimensions.<n>Our analyses reveal distinctive organizational patterns as well as persistent gaps in current review practices.<n>Finally, our evaluation of state-of-the-art AI-generated reviews indicates encouraging advances in coherence and organization.
  arXiv  Detail & Related papers  (2024-02-20T11:28:50Z)
• Learning by Self-Explaining [23.420673675343266]
  We introduce a novel workflow in the context of image classification, termed Learning by Self-Explaining (LSX) LSX utilizes aspects of self-refining AI and human-guided explanatory machine learning. Our results indicate improvements via Learning by Self-Explaining on several levels.
  arXiv  Detail & Related papers  (2023-09-15T13:41:57Z)
• Analysis of Learner Independent Variables for Estimating Assessment Items Difficulty Level [0.0]
  This research proposes a model for predicting the difficulty level of assessment items in engineering courses. Experimental results from three engineering courses provide around 80% accuracy in classification of items using the proposed model.
  arXiv  Detail & Related papers  (2022-06-09T11:01:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.