Difficulty-Focused Contrastive Learning for Knowledge Tracing with a Large Language Model-Based Difficulty Prediction

• URL: http://arxiv.org/abs/2312.11890v1
• Date: Tue, 19 Dec 2023 06:26:25 GMT
• Title: Difficulty-Focused Contrastive Learning for Knowledge Tracing with a Large Language Model-Based Difficulty Prediction
• Authors: Unggi Lee, Sungjun Yoon, Joon Seo Yun, Kyoungsoo Park, YoungHoon Jung, Damji Stratton, Hyeoncheol Kim
• Abstract summary: This paper presents novel techniques for enhancing the performance of knowledge tracing (KT) models by focusing on the crucial factor of question and concept difficulty level. We propose a difficulty-centered contrastive learning method for KT models and a Large Language Model (LLM)-based framework for difficulty prediction.
• Score: 2.8946115982002443
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper presents novel techniques for enhancing the performance of knowledge tracing (KT) models by focusing on the crucial factor of question and concept difficulty level. Despite the acknowledged significance of difficulty, previous KT research has yet to exploit its potential for model optimization and has struggled to predict difficulty from unseen data. To address these problems, we propose a difficulty-centered contrastive learning method for KT models and a Large Language Model (LLM)-based framework for difficulty prediction. These innovative methods seek to improve the performance of KT models and provide accurate difficulty estimates for unseen data. Our ablation study demonstrates the efficacy of these techniques by demonstrating enhanced KT model performance. Nonetheless, the complex relationship between language and difficulty merits further investigation.

Related papers

• Fairness and Robustness in Machine Unlearning [20.758637391023345]
  We focus on fairness and robustness in machine unlearning algorithms. Experiments demonstrate the vulnerability of current state-of-the-art approximated unlearning algorithms to adversarial attacks. We demonstrate that unlearning in the intermediate and last layers is sufficient and cost-effective for time and memory complexity.
  arXiv  Detail & Related papers  (2025-04-18T10:31:44Z)
• AdvKT: An Adversarial Multi-Step Training Framework for Knowledge Tracing [64.79967583649407]
  Knowledge Tracing (KT) monitors students' knowledge states and simulates their responses to question sequences. Existing KT models typically follow a single-step training paradigm, which leads to significant error accumulation. We propose a novel Adversarial Multi-Step Training Framework for Knowledge Tracing (AdvKT) which focuses on the multi-step KT task.
  arXiv  Detail & Related papers  (2025-04-07T03:31:57Z)
• LLM-driven Effective Knowledge Tracing by Integrating Dual-channel Difficulty [9.683271515093994]
  We propose a novel Dual-channel Difficulty-aware Knowledge Tracing (DDKT) framework. It incorporates difficulty bias-aware algorithms and student mastery algorithms for precise difficulty measurement. Our framework introduces three key innovations: (1) Difficulty Balance Perception Sequence (DBPS) - students' subjective perceptions combined with objective difficulty, measuring gaps between LLM-assessed difficulty, mathematical-statistical difficulty, and students' subjective perceived difficulty through attention mechanisms; (2) Difficulty Mastery Ratio (DMR) - precise modeling of student mastery levels through different difficulty zones; and (3) Knowledge State Update Mechanism - implementing personalized knowledge acquisition through gated
  arXiv  Detail & Related papers  (2025-02-27T09:36:27Z)
• Exploring the Potential of Large Language Models for Estimating the Reading Comprehension Question Difficulty [2.335292678914151]
  This study investigates the effectiveness of Large Language Models (LLMs) in estimating the difficulty of reading comprehension questions. We use OpenAI's GPT-4o and o1, in estimating the difficulty of reading comprehension questions using the Study Aid and Reading Assessment (SARA) dataset. The results indicate that while the models yield difficulty estimates that align meaningfully with derived IRT parameters, there are notable differences in their sensitivity to extreme item characteristics.
  arXiv  Detail & Related papers  (2025-02-25T02:28:48Z)
• DASKT: A Dynamic Affect Simulation Method for Knowledge Tracing [51.665582274736785]
  Knowledge Tracing (KT) predicts future performance by students' historical computation, and understanding students' affective states can enhance the effectiveness of KT. We propose Affect Dynamic Knowledge Tracing (DASKT) to explore the impact of various student affective states on their knowledge states. Our research highlights a promising avenue for future studies, focusing on achieving high interpretability and accuracy.
  arXiv  Detail & Related papers  (2025-01-18T10:02:10Z)
• How Hard is this Test Set? NLI Characterization by Exploiting Training Dynamics [49.9329723199239]
  We propose a method for the automated creation of a challenging test set without relying on the manual construction of artificial and unrealistic examples. We categorize the test set of popular NLI datasets into three difficulty levels by leveraging methods that exploit training dynamics. When our characterization method is applied to the training set, models trained with only a fraction of the data achieve comparable performance to those trained on the full dataset.
  arXiv  Detail & Related papers  (2024-10-04T13:39:21Z)
• Enhancing Training Data Attribution for Large Language Models with Fitting Error Consideration [74.09687562334682]
  We introduce a novel training data attribution method called Debias and Denoise Attribution (DDA) Our method significantly outperforms existing approaches, achieving an averaged AUC of 91.64%. DDA exhibits strong generality and scalability across various sources and different-scale models like LLaMA2, QWEN2, and Mistral.
  arXiv  Detail & Related papers  (2024-10-02T07:14:26Z)
• Scaling Vision-Language Models with Sparse Mixture of Experts [128.0882767889029]
  We show that mixture-of-experts (MoE) techniques can achieve state-of-the-art performance on a range of benchmarks over dense models of equivalent computational cost. Our research offers valuable insights into stabilizing the training of MoE models, understanding the impact of MoE on model interpretability, and balancing the trade-offs between compute performance when scaling vision-language models.
  arXiv  Detail & Related papers  (2023-03-13T16:00:31Z)
• GLUECons: A Generic Benchmark for Learning Under Constraints [102.78051169725455]
  In this work, we create a benchmark that is a collection of nine tasks in the domains of natural language processing and computer vision. We model external knowledge as constraints, specify the sources of the constraints for each task, and implement various models that use these constraints.
  arXiv  Detail & Related papers  (2023-02-16T16:45:36Z)
• Improving Interpretability of Deep Sequential Knowledge Tracing Models with Question-centric Cognitive Representations [22.055683237994696]
  We present QIKT, a question-centric interpretable KT model to address the above challenges. The proposed QIKT approach explicitly models students' knowledge state variations at a fine-grained level. It outperforms a wide range of deep learning based KT models in terms of prediction accuracy with better model interpretability.
  arXiv  Detail & Related papers  (2023-02-14T08:14:30Z)
• Improving Sample Efficiency of Deep Learning Models in Electricity Market [0.41998444721319217]
  We propose a general framework, namely Knowledge-Augmented Training (KAT), to improve the sample efficiency. We propose a novel data augmentation technique to generate some synthetic data, which are later processed by an improved training strategy. Modern learning theories demonstrate the effectiveness of our method in terms of effective prediction error feedbacks, a reliable loss function, and rich gradient noises.
  arXiv  Detail & Related papers  (2022-10-11T16:35:13Z)
• What Makes Good Contrastive Learning on Small-Scale Wearable-based Tasks? [59.51457877578138]
  We study contrastive learning on the wearable-based activity recognition task. This paper presents an open-source PyTorch library textttCL-HAR, which can serve as a practical tool for researchers.
  arXiv  Detail & Related papers  (2022-02-12T06:10:15Z)
• An Empirical Comparison of Deep Learning Models for Knowledge Tracing on Large-Scale Dataset [10.329254031835953]
  Knowledge tracing is a problem of modeling each student's mastery of knowledge concepts. Recent release of large-scale student performance dataset citechoi 2019ednet motivates the analysis of performance of deep learning approaches.
  arXiv  Detail & Related papers  (2021-01-16T04:58:17Z)
• Towards Interpretable Deep Learning Models for Knowledge Tracing [62.75876617721375]
  We propose to adopt the post-hoc method to tackle the interpretability issue for deep learning based knowledge tracing (DLKT) models. Specifically, we focus on applying the layer-wise relevance propagation (LRP) method to interpret RNN-based DLKT model. Experiment results show the feasibility using the LRP method for interpreting the DLKT model's predictions.
  arXiv  Detail & Related papers  (2020-05-13T04:03:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.