Interpretable Difficulty-Aware Knowledge Tracing in Tutor-Student Dialogues

• URL: http://arxiv.org/abs/2605.01097v1
• Date: Fri, 01 May 2026 21:07:24 GMT
• Title: Interpretable Difficulty-Aware Knowledge Tracing in Tutor-Student Dialogues
• Abstract summary: We propose an interpretable difficulty-aware conversational KT framework built upon large language models (LLMs)<n>The framework incorporates the original textual question and the next tutor-posed task to estimate the student's knowledge state and the difficulty of the upcoming turn.<n>We evaluate the framework on two tutor-student dialogue datasets.
• Score: 46.915280360954576
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances in large language models (LLMs) have led to the development of AI-powered tutoring systems that provide interactive support via dialogue. To enable these tutoring systems to provide personalized support, it is essential to assess student performance at each turn, motivating knowledge tracing (KT) in dialogue settings. However, existing dialogue-based KT approaches often ignore question difficulty modeling and rely on opaque latent representations from LLMs, hindering accurate and interpretable prediction. In this work, we propose an interpretable difficulty-aware conversational KT framework built upon LLMs, which explicitly models students' abilities and the difficulty of tutor-posed tasks at each turn. The framework incorporates the original textual question and the next tutor-posed task to estimate the student's knowledge state and the difficulty of the upcoming turn. Furthermore, it integrates Item Response Theory to map LLM's outputs into student ability and question difficulty parameters, enabling interpretable prediction of student performance grounded in cognitive theories of learning. We evaluate the framework on two tutor-student dialogue datasets. Both quantitative and qualitative results show that our framework outperforms existing KT baselines, meanwhile generating interpretable outputs consistent with cognitive theory.

Related papers

• Designing Gaze Analytics for ELA Instruction: A User-Centered Dashboard with Conversational AI Support [3.741199946315248]
  Eye-tracking offers rich insights into student cognition and engagement.<n>However, it remains underutilized in classroom-facing educational technology.<n>We present the iterative design and evaluation of a gaze-based learning analytics dashboard for English Language Arts.
  arXiv  Detail & Related papers  (2025-09-03T22:01:14Z)
• Personalizing Student-Agent Interactions Using Log-Contextualized Retrieval Augmented Generation (RAG) [3.5262811753348715]
  Large language models (LLMs) facilitate dynamic pedagogical interactions but undermine confidence, trust, and instructional value.<n>Retrieval-augmented generation (RAG) grounds LLM outputs in curated knowledge but requires a clear semantic link between user input and a knowledge base.<n>We propose log-contextualized RAG (LC-RAG), which enhances RAG retrieval by using environment logs to contextualize collaborative discourse.
  arXiv  Detail & Related papers  (2025-05-22T19:31:40Z)
• Alignment Drift in CEFR-prompted LLMs for Interactive Spanish Tutoring [0.0]
  This paper investigates the potentials of Large Language Models (LLMs) as adaptive tutors in the context of second-language learning.<n>We simulate full teacher-student dialogues in Spanish using instruction-tuned, open-source LLMs ranging in size from 7B to 12B parameters.<n>The output from the tutor model is then used to evaluate the effectiveness of CEFR-based prompting to control text difficulty across three proficiency levels.
  arXiv  Detail & Related papers  (2025-05-13T08:50:57Z)
• INTERACT: Enabling Interactive, Question-Driven Learning in Large Language Models [15.825663946923289]
  Large language models (LLMs) excel at answering questions but remain passive learners-absorbing static data without the ability to question and refine knowledge.<n>This paper explores how LLMs can transition to interactive, question-driven learning through student-teacher dialogues.
  arXiv  Detail & Related papers  (2024-12-16T02:28:53Z)
• Exploring Knowledge Tracing in Tutor-Student Dialogues using LLMs [49.18567856499736]
  We investigate whether large language models (LLMs) can be supportive of open-ended dialogue tutoring.<n>We apply a range of knowledge tracing (KT) methods on the resulting labeled data to track student knowledge levels over an entire dialogue.<n>We conduct experiments on two tutoring dialogue datasets, and show that a novel yet simple LLM-based method, LLMKT, significantly outperforms existing KT methods in predicting student response correctness in dialogues.
  arXiv  Detail & Related papers  (2024-09-24T22:31:39Z)
• SINKT: A Structure-Aware Inductive Knowledge Tracing Model with Large Language Model [64.92472567841105]
  Knowledge Tracing (KT) aims to determine whether students will respond correctly to the next question. Structure-aware Inductive Knowledge Tracing model with large language model (dubbed SINKT) SINKT predicts the student's response to the target question by interacting with the student's knowledge state and the question representation.
  arXiv  Detail & Related papers  (2024-07-01T12:44:52Z)
• Student Data Paradox and Curious Case of Single Student-Tutor Model: Regressive Side Effects of Training LLMs for Personalized Learning [25.90420385230675]
  The pursuit of personalized education has led to the integration of Large Language Models (LLMs) in developing intelligent tutoring systems. Our research uncovers a fundamental challenge in this approach: the Student Data Paradox'' This paradox emerges when LLMs, trained on student data to understand learner behavior, inadvertently compromise their own factual knowledge and reasoning abilities.
  arXiv  Detail & Related papers  (2024-04-23T15:57:55Z)
• Exploring the Factual Consistency in Dialogue Comprehension of Large Language Models [51.75805497456226]
  This work focuses on the factual consistency issue with the help of the dialogue summarization task. Our evaluation shows that, on average, 26.8% of the summaries generated by LLMs contain factual inconsistency. To stimulate and enhance the dialogue comprehension ability of LLMs, we propose a fine-tuning paradigm with auto-constructed multi-task data.
  arXiv  Detail & Related papers  (2023-11-13T09:32:12Z)
• Improving Open Information Extraction with Large Language Models: A Study on Demonstration Uncertainty [52.72790059506241]
  Open Information Extraction (OIE) task aims at extracting structured facts from unstructured text. Despite the potential of large language models (LLMs) like ChatGPT as a general task solver, they lag behind state-of-the-art (supervised) methods in OIE tasks.
  arXiv  Detail & Related papers  (2023-09-07T01:35:24Z)
• Opportunities and Challenges in Neural Dialog Tutoring [54.07241332881601]
  We rigorously analyze various generative language models on two dialog tutoring datasets for language learning. We find that although current approaches can model tutoring in constrained learning scenarios, they perform poorly in less constrained scenarios. Our human quality evaluation shows that both models and ground-truth annotations exhibit low performance in terms of equitable tutoring.
  arXiv  Detail & Related papers  (2023-01-24T11:00:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.