Beyond Right and Wrong: Mitigating Cold Start in Knowledge Tracing Using Large Language Model and Option Weight

• URL: http://arxiv.org/abs/2410.12872v1
• Date: Mon, 14 Oct 2024 16:25:48 GMT
• Title: Beyond Right and Wrong: Mitigating Cold Start in Knowledge Tracing Using Large Language Model and Option Weight
• Authors: JongWoo Kim, SeongYeub Chu, Bryan Wong, Mun Yi,
• Abstract summary: Knowledge Tracing (KT) is vital in educational data mining, enabling personalized learning. This study introduces the LOKT (Large Language Model Option-weighted Knowledge Tracing) model to address the cold start problem.
• Score: 0.14999444543328289
• License:
• Abstract: Knowledge Tracing (KT) is vital in educational data mining, enabling personalized learning by tracking learners' knowledge states and forecasting their academic outcomes. This study introduces the LOKT (Large Language Model Option-weighted Knowledge Tracing) model to address the cold start problem where limited historical data available using large language models (LLMs). While traditional KT models have incorporated option weights, our research extends this by integrating these weights into an LLM-based KT framework. Moving beyond the binary classification of correct and incorrect responses, we emphasize that different types of incorrect answers offer valuable insights into a learner's knowledge state. By converting these responses into text-based ordinal categories, we enable LLMs to assess learner understanding with greater clarity, although our approach focuses on the final knowledge state rather than the progression of learning over time. Using five public datasets, we demonstrate that the LOKT model sustains high predictive accuracy even with limited data, effectively addressing both "learner cold-start" and "system cold-start" scenarios. These findings showcase LOKT's potential to enhance LLM-based learning tools and support early-stage personalization.

Related papers

• CLST: Cold-Start Mitigation in Knowledge Tracing by Aligning a Generative Language Model as a Students' Knowledge Tracer [1.6713666776851528]
  We propose cold-start mitigation in knowledge tracing by aligning a generative language model as a students' knowledge tracer (T) We framed the KT task as a natural language processing task, wherein problem-solving data are expressed in natural language. We evaluated the performance of the CLST in situations of data scarcity using various baseline models for comparison.
  arXiv  Detail & Related papers  (2024-06-13T09:21:43Z)
• Language Model Can Do Knowledge Tracing: Simple but Effective Method to Integrate Language Model and Knowledge Tracing Task [3.1459398432526267]
  This paper proposes Language model-based Knowledge Tracing (LKT), a novel framework that integrates pre-trained language models (PLMs) with Knowledge Tracing methods. LKT effectively incorporates textual information and significantly outperforms previous KT models on large benchmark datasets.
  arXiv  Detail & Related papers  (2024-06-05T03:26:59Z)
• Supervised Knowledge Makes Large Language Models Better In-context Learners [94.89301696512776]
  Large Language Models (LLMs) exhibit emerging in-context learning abilities through prompt engineering. The challenge of improving the generalizability and factuality of LLMs in natural language understanding and question answering remains under-explored. We propose a framework that enhances the reliability of LLMs as it: 1) generalizes out-of-distribution data, 2) elucidates how LLMs benefit from discriminative models, and 3) minimizes hallucinations in generative tasks.
  arXiv  Detail & Related papers  (2023-12-26T07:24:46Z)
• From Supervised to Generative: A Novel Paradigm for Tabular Deep Learning with Large Language Models [18.219485459836285]
  Generative Tabular Learning (GTL) is a novel framework that integrates the advanced functionalities of large language models (LLMs) Our empirical study spans 384 public datasets, rigorously analyzing GTL's scaling behaviors. GTL-LLaMA-2 model demonstrates superior zero-shot and in-context learning capabilities across numerous classification and regression tasks.
  arXiv  Detail & Related papers  (2023-10-11T09:37:38Z)
• Give Us the Facts: Enhancing Large Language Models with Knowledge Graphs for Fact-aware Language Modeling [34.59678835272862]
  ChatGPT, a representative large language model (LLM), has gained considerable attention due to its powerful emergent abilities. This paper proposes to enhance LLMs with knowledge graph-enhanced large language models (KGLLMs) KGLLM provides a solution to enhance LLMs' factual reasoning ability, opening up new avenues for LLM research.
  arXiv  Detail & Related papers  (2023-06-20T12:21:06Z)
• Self-Checker: Plug-and-Play Modules for Fact-Checking with Large Language Models [75.75038268227554]
  Self-Checker is a framework comprising a set of plug-and-play modules that facilitate fact-checking. This framework provides a fast and efficient way to construct fact-checking systems in low-resource environments.
  arXiv  Detail & Related papers  (2023-05-24T01:46:07Z)
• Temporal Knowledge Graph Forecasting Without Knowledge Using In-Context Learning [23.971206470486468]
  We present a framework that converts relevant historical facts into prompts and generates ranked predictions using token probabilities. Surprisingly, we observe that LLMs, out-of-the-box, perform on par with state-of-the-art TKG models. We also discover that using numerical indices instead of entity/relation names, does not significantly affect the performance.
  arXiv  Detail & Related papers  (2023-05-17T23:50:28Z)
• Large Language Models Are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context Learning [104.58874584354787]
  In recent years, pre-trained large language models (LLMs) have demonstrated remarkable efficiency in achieving an inference-time few-shot learning capability known as in-context learning. This study aims to examine the in-context learning phenomenon through a Bayesian lens, viewing real-world LLMs as latent variable models.
  arXiv  Detail & Related papers  (2023-01-27T18:59:01Z)
• Large Language Models with Controllable Working Memory [64.71038763708161]
  Large language models (LLMs) have led to a series of breakthroughs in natural language processing (NLP) What further sets these models apart is the massive amounts of world knowledge they internalize during pretraining. How the model's world knowledge interacts with the factual information presented in the context remains under explored.
  arXiv  Detail & Related papers  (2022-11-09T18:58:29Z)
• Knowledge-driven Active Learning [70.37119719069499]
  Active learning strategies aim at minimizing the amount of labelled data required to train a Deep Learning model. Most active strategies are based on uncertain sample selection, and even often restricted to samples lying close to the decision boundary. Here we propose to take into consideration common domain-knowledge and enable non-expert users to train a model with fewer samples.
  arXiv  Detail & Related papers  (2021-10-15T06:11:53Z)
• Towards Interpretable Deep Learning Models for Knowledge Tracing [62.75876617721375]
  We propose to adopt the post-hoc method to tackle the interpretability issue for deep learning based knowledge tracing (DLKT) models. Specifically, we focus on applying the layer-wise relevance propagation (LRP) method to interpret RNN-based DLKT model. Experiment results show the feasibility using the LRP method for interpreting the DLKT model's predictions.
  arXiv  Detail & Related papers  (2020-05-13T04:03:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.