Consistency and Monotonicity Regularization for Neural Knowledge Tracing

• URL: http://arxiv.org/abs/2105.00607v1
• Date: Mon, 3 May 2021 02:36:29 GMT
• Title: Consistency and Monotonicity Regularization for Neural Knowledge Tracing
• Authors: Seewoo Lee, Youngduck Choi, Juneyoung Park, Byungsoo Kim and Jinwoo Shin
• Abstract summary: Knowledge Tracing (KT) tracking a human's knowledge acquisition is a central component in online learning and AI in Education. We propose three types of novel data augmentation, coined replacement, insertion, and deletion, along with corresponding regularization losses. Extensive experiments on various KT benchmarks show that our regularization scheme consistently improves the model performances.
• Score: 50.92661409499299
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Knowledge Tracing (KT), tracking a human's knowledge acquisition, is a central component in online learning and AI in Education. In this paper, we present a simple, yet effective strategy to improve the generalization ability of KT models: we propose three types of novel data augmentation, coined replacement, insertion, and deletion, along with corresponding regularization losses that impose certain consistency or monotonicity biases on the model's predictions for the original and augmented sequence. Extensive experiments on various KT benchmarks show that our regularization scheme consistently improves the model performances, under 3 widely-used neural networks and 4 public benchmarks, e.g., it yields 6.3% improvement in AUC under the DKT model and the ASSISTmentsChall dataset.

Related papers

• The Surprising Effectiveness of Test-Time Training for Abstract Reasoning [64.36534512742736]
  We investigate the effectiveness of test-time training (TTT) as a mechanism for improving models' reasoning capabilities. TTT significantly improves performance on ARC tasks, achieving up to 6x improvement in accuracy compared to base fine-tuned models. Our findings suggest that explicit symbolic search is not the only path to improved abstract reasoning in neural language models.
  arXiv  Detail & Related papers  (2024-11-11T18:59:45Z)
• Contrastive Learning for Character Detection in Ancient Greek Papyri [0.6361669177741777]
  This thesis investigates the effectiveness of SimCLR, a contrastive learning technique, in Greek letter recognition. Pretraining of SimCLR is conducted on the Alpub dataset, followed by fine-tuning on the ICDAR dataset. Our experiments show that SimCLR does not outperform the baselines in letter recognition tasks.
  arXiv  Detail & Related papers  (2024-09-16T10:41:29Z)
• On Machine Learning Approaches for Protein-Ligand Binding Affinity Prediction [2.874893537471256]
  This study evaluates the performance of classical tree-based models and advanced neural networks in protein-ligand binding affinity prediction. We show that combining 2D and 3D model strengths improves active learning outcomes beyond current state-of-the-art approaches.
  arXiv  Detail & Related papers  (2024-07-15T13:06:00Z)
• Enhancing Robustness of Vision-Language Models through Orthogonality Learning and Self-Regularization [77.62516752323207]
  We introduce an orthogonal fine-tuning method for efficiently fine-tuning pretrained weights and enabling enhanced robustness and generalization. A self-regularization strategy is further exploited to maintain the stability in terms of zero-shot generalization of VLMs, dubbed OrthSR. For the first time, we revisit the CLIP and CoOp with our method to effectively improve the model on few-shot image classficiation scenario.
  arXiv  Detail & Related papers  (2024-07-11T10:35:53Z)
• CuSINeS: Curriculum-driven Structure Induced Negative Sampling for Statutory Article Retrieval [1.3723120574076126]
  CuSINeS is a negative sampling approach to enhance the performance of Statutory Article Retrieval (SAR) It employs a curriculum-based negative sampling strategy guiding the model to focus on easier negatives. It also leverages the hierarchical and sequential information derived from the structural organization of statutes to evaluate the difficulty of samples.
  arXiv  Detail & Related papers  (2024-03-31T07:49:23Z)
• DiTMoS: Delving into Diverse Tiny-Model Selection on Microcontrollers [34.282971510732736]
  We introduce DiTMoS, a novel DNN training and inference framework with a selector-classifiers architecture. A composition of weak models can exhibit high diversity and the union of them can significantly boost the accuracy upper bound. We deploy DiTMoS on the Neucleo STM32F767ZI board and evaluate it based on three time-series datasets for human activity recognition, keywords spotting, and emotion recognition.
  arXiv  Detail & Related papers  (2024-03-14T02:11:38Z)
• TWINS: A Fine-Tuning Framework for Improved Transferability of Adversarial Robustness and Generalization [89.54947228958494]
  This paper focuses on the fine-tuning of an adversarially pre-trained model in various classification tasks. We propose a novel statistics-based approach, Two-WIng NormliSation (TWINS) fine-tuning framework. TWINS is shown to be effective on a wide range of image classification datasets in terms of both generalization and robustness.
  arXiv  Detail & Related papers  (2023-03-20T14:12:55Z)
• An Empirical Study on Distribution Shift Robustness From the Perspective of Pre-Training and Data Augmentation [91.62129090006745]
  This paper studies the distribution shift problem from the perspective of pre-training and data augmentation. We provide the first comprehensive empirical study focusing on pre-training and data augmentation.
  arXiv  Detail & Related papers  (2022-05-25T13:04:53Z)
• Enhancing Knowledge Tracing via Adversarial Training [5.461665809706664]
  We study the problem of knowledge tracing (KT) where the goal is to trace the students' knowledge mastery over time. Recent advances on KT have increasingly concentrated on exploring deep neural networks (DNNs) to improve the performance of KT. We propose an efficient AT based KT method (ATKT) to enhance KT model's generalization and thus push the limit of KT.
  arXiv  Detail & Related papers  (2021-08-10T03:35:13Z)
• CASTLE: Regularization via Auxiliary Causal Graph Discovery [89.74800176981842]
  We introduce Causal Structure Learning (CASTLE) regularization and propose to regularize a neural network by jointly learning the causal relationships between variables. CASTLE efficiently reconstructs only the features in the causal DAG that have a causal neighbor, whereas reconstruction-based regularizers suboptimally reconstruct all input features.
  arXiv  Detail & Related papers  (2020-09-28T09:49:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.