A Systematic Review of Knowledge Tracing and Large Language Models in Education: Opportunities, Issues, and Future Research

• URL: http://arxiv.org/abs/2412.09248v1
• Date: Thu, 12 Dec 2024 13:00:50 GMT
• Title: A Systematic Review of Knowledge Tracing and Large Language Models in Education: Opportunities, Issues, and Future Research
• Authors: Yongwan Cho, Rabia Emhamed AlMamlook, Tasnim Gharaibeh,
• Abstract summary: Large Language Models (LLMs) are pre-trained on vast natural language datasets. This systematic review explores the intersections, opportunities, and challenges of combining KT models and LLMs in educational contexts.
• Score: 0.0
• License:
• Abstract: Knowledge Tracing (KT) is a research field that aims to estimate a student's knowledge state through learning interactions-a crucial component of Intelligent Tutoring Systems (ITSs). Despite significant advancements, no current KT models excel in both predictive accuracy and interpretability. Meanwhile, Large Language Models (LLMs), pre-trained on vast natural language datasets, have emerged as powerful tools with immense potential in various educational applications. This systematic review explores the intersections, opportunities, and challenges of combining KT models and LLMs in educational contexts. The review first investigates LLM applications in education, including their adaptability to domain-specific content and ability to support personalized learning. It then examines the development and current state of KT models, from traditional to advanced approaches, aiming to uncover potential challenges that LLMs could mitigate. The core of this review focuses on integrating LLMs with KT, exploring three primary functions: addressing general concerns in KT fields, overcoming specific KT model limitations, and performing as KT models themselves. Our findings reveal that LLMs can be customized for specific educational tasks through tailor-making techniques such as in-context learning and agent-based approaches, effectively managing complex and unbalanced educational data. These models can enhance existing KT models' performance and solve cold-start problems by generating relevant features from question data. However, both current models depend heavily on structured, limited datasets, missing opportunities to use diverse educational data that could offer deeper insights into individual learners and support various educational settings.

Related papers

• The Inherent Limits of Pretrained LLMs: The Unexpected Convergence of Instruction Tuning and In-Context Learning Capabilities [51.594836904623534]
  We investigate whether instruction-tuned models possess fundamentally different capabilities from base models that are prompted using in-context examples. We show that the performance of instruction-tuned models is significantly correlated with the in-context performance of their base counterparts. Specifically, we extend this understanding to instruction-tuned models, suggesting that their pretraining data similarly sets a limiting boundary on the tasks they can solve.
  arXiv  Detail & Related papers  (2025-01-15T10:57:55Z)
• Beyond Right and Wrong: Mitigating Cold Start in Knowledge Tracing Using Large Language Model and Option Weight [0.14999444543328289]
  Knowledge Tracing (KT) is vital in educational data mining, enabling personalized learning. This study introduces the LOKT (Large Language Model Option-weighted Knowledge Tracing) model to address the cold start problem.
  arXiv  Detail & Related papers  (2024-10-14T16:25:48Z)
• SINKT: A Structure-Aware Inductive Knowledge Tracing Model with Large Language Model [64.92472567841105]
  Knowledge Tracing (KT) aims to determine whether students will respond correctly to the next question. Structure-aware Inductive Knowledge Tracing model with large language model (dubbed SINKT) SINKT predicts the student's response to the target question by interacting with the student's knowledge state and the question representation.
  arXiv  Detail & Related papers  (2024-07-01T12:44:52Z)
• Can Long-Context Language Models Subsume Retrieval, RAG, SQL, and More? [54.667202878390526]
  Long-context language models (LCLMs) have the potential to revolutionize our approach to tasks traditionally reliant on external tools like retrieval systems or databases. We introduce LOFT, a benchmark of real-world tasks requiring context up to millions of tokens designed to evaluate LCLMs' performance on in-context retrieval and reasoning. Our findings reveal LCLMs' surprising ability to rival state-of-the-art retrieval and RAG systems, despite never having been explicitly trained for these tasks.
  arXiv  Detail & Related papers  (2024-06-19T00:28:58Z)
• CLST: Cold-Start Mitigation in Knowledge Tracing by Aligning a Generative Language Model as a Students' Knowledge Tracer [1.6713666776851528]
  We propose cold-start mitigation in knowledge tracing by aligning a generative language model as a students' knowledge tracer (T) We framed the KT task as a natural language processing task, wherein problem-solving data are expressed in natural language. We evaluated the performance of the CLST in situations of data scarcity using various baseline models for comparison.
  arXiv  Detail & Related papers  (2024-06-13T09:21:43Z)
• Language Model Can Do Knowledge Tracing: Simple but Effective Method to Integrate Language Model and Knowledge Tracing Task [3.1459398432526267]
  This paper proposes Language model-based Knowledge Tracing (LKT), a novel framework that integrates pre-trained language models (PLMs) with Knowledge Tracing methods. LKT effectively incorporates textual information and significantly outperforms previous KT models on large benchmark datasets.
  arXiv  Detail & Related papers  (2024-06-05T03:26:59Z)
• Improving Low-Resource Knowledge Tracing Tasks by Supervised Pre-training and Importance Mechanism Fine-tuning [25.566963415155325]
  We propose a low-resource KT framework called LoReKT to address above challenges. Inspired by the prevalent "pre-training and fine-tuning" paradigm, we aim to learn transferable parameters and representations from rich-resource KT datasets. We design an encoding mechanism to incorporate student interactions from multiple KT data sources.
  arXiv  Detail & Related papers  (2024-03-11T13:44:43Z)
• Towards Modeling Learner Performance with Large Language Models [7.002923425715133]
  This paper investigates whether the pattern recognition and sequence modeling capabilities of LLMs can be extended to the domain of knowledge tracing. We compare two approaches to using LLMs for this task, zero-shot prompting and model fine-tuning, with existing, non-LLM approaches to knowledge tracing. While LLM-based approaches do not achieve state-of-the-art performance, fine-tuned LLMs surpass the performance of naive baseline models and perform on par with standard Bayesian Knowledge Tracing approaches.
  arXiv  Detail & Related papers  (2024-02-29T14:06:34Z)
• A Survey on Knowledge Distillation of Large Language Models [99.11900233108487]
  Knowledge Distillation (KD) emerges as a pivotal methodology for transferring advanced capabilities to open-source models. This paper presents a comprehensive survey of KD's role within the realm of Large Language Models (LLMs)
  arXiv  Detail & Related papers  (2024-02-20T16:17:37Z)
• Large Language Models with Controllable Working Memory [64.71038763708161]
  Large language models (LLMs) have led to a series of breakthroughs in natural language processing (NLP) What further sets these models apart is the massive amounts of world knowledge they internalize during pretraining. How the model's world knowledge interacts with the factual information presented in the context remains under explored.
  arXiv  Detail & Related papers  (2022-11-09T18:58:29Z)
• A Survey of Knowledge Tracing: Models, Variants, and Applications [70.69281873057619]
  Knowledge Tracing is one of the fundamental tasks for student behavioral data analysis. We present three types of fundamental KT models with distinct technical routes. We discuss potential directions for future research in this rapidly growing field.
  arXiv  Detail & Related papers  (2021-05-06T13:05:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.