Domain Generalizable Knowledge Tracing via Concept Aggregation and Relation-Based Attention

• URL: http://arxiv.org/abs/2407.02547v1
• Date: Tue, 2 Jul 2024 13:13:44 GMT
• Title: Domain Generalizable Knowledge Tracing via Concept Aggregation and Relation-Based Attention
• Authors: Yuquan Xie, Wanqi Yang, Jinyu Wei, Ming Yang, Yang Gao,
• Abstract summary: We propose a domain generalization approach for knowledge tracing, where existing education systems are considered source domains. We also present a concept aggregation approach designed to reduce conceptual disparities within sequences of student interactions from diverse domains. To fully leverage exercise information, we propose a new knowledge tracing model tailored for the domain generalization KT task, named Domain-General Relation-based Knowledge Tracing (DGRKT)
• Score: 10.95112067894146
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Knowledge Tracing (KT) is a critical task in online education systems, aiming to monitor students' knowledge states throughout a learning period. Common KT approaches involve predicting the probability of a student correctly answering the next question based on their exercise history. However, these methods often suffer from performance degradation when faced with the scarcity of student interactions in new education systems. To address this, we leverage student interactions from existing education systems to mitigate performance degradation caused by limited training data. Nevertheless, these interactions exhibit significant differences since they are derived from different education systems. To address this issue, we propose a domain generalization approach for knowledge tracing, where existing education systems are considered source domains, and new education systems with limited data are considered target domains. Additionally, we design a domain-generalizable knowledge tracing framework (DGKT) that can be applied to any KT model. Specifically, we present a concept aggregation approach designed to reduce conceptual disparities within sequences of student interactions from diverse domains. To further mitigate domain discrepancies, we introduce a novel normalization module called Sequence Instance Normalization (SeqIN). Moreover, to fully leverage exercise information, we propose a new knowledge tracing model tailored for the domain generalization KT task, named Domain-Generalizable Relation-based Knowledge Tracing (DGRKT). Extensive experiments across five benchmark datasets demonstrate that the proposed method performs well despite limited training data.

Related papers

• Mind the Interference: Retaining Pre-trained Knowledge in Parameter Efficient Continual Learning of Vision-Language Models [79.28821338925947]
  Domain-Class Incremental Learning is a realistic but challenging continual learning scenario. To handle these diverse tasks, pre-trained Vision-Language Models (VLMs) are introduced for their strong generalizability. This incurs a new problem: the knowledge encoded in the pre-trained VLMs may be disturbed when adapting to new tasks, compromising their inherent zero-shot ability. Existing methods tackle it by tuning VLMs with knowledge distillation on extra datasets, which demands heavy overhead. We propose the Distribution-aware Interference-free Knowledge Integration (DIKI) framework, retaining pre-trained knowledge of
  arXiv  Detail & Related papers  (2024-07-07T12:19:37Z)
• Domain Generalization through Meta-Learning: A Survey [6.524870790082051]
  Deep neural networks (DNNs) have revolutionized artificial intelligence but often lack performance when faced with out-of-distribution (OOD) data. This survey paper delves into the realm of meta-learning with a focus on its contribution to domain generalization.
  arXiv  Detail & Related papers  (2024-04-03T14:55:17Z)
• Direct Distillation between Different Domains [97.39470334253163]
  We propose a new one-stage method dubbed Direct Distillation between Different Domains" (4Ds) We first design a learnable adapter based on the Fourier transform to separate the domain-invariant knowledge from the domain-specific knowledge. We then build a fusion-activation mechanism to transfer the valuable domain-invariant knowledge to the student network.
  arXiv  Detail & Related papers  (2024-01-12T02:48:51Z)
• Improving Intrusion Detection with Domain-Invariant Representation Learning in Latent Space [4.871119861180455]
  We introduce a two-phase representation learning technique using multi-task learning. We disentangle the latent space by minimizing the mutual information between the prior and latent space. We assess the model's efficacy across multiple cybersecurity datasets.
  arXiv  Detail & Related papers  (2023-12-28T17:24:13Z)
• A Recent Survey of Heterogeneous Transfer Learning [15.830786437956144]
  heterogeneous transfer learning has become a vital strategy in various tasks. We offer an extensive review of over 60 HTL methods, covering both data-based and model-based approaches. We explore applications in natural language processing, computer vision, multimodal learning, and biomedicine.
  arXiv  Detail & Related papers  (2023-10-12T16:19:58Z)
• Prior Knowledge Guided Unsupervised Domain Adaptation [82.9977759320565]
  We propose a Knowledge-guided Unsupervised Domain Adaptation (KUDA) setting where prior knowledge about the target class distribution is available. In particular, we consider two specific types of prior knowledge about the class distribution in the target domain: Unary Bound and Binary Relationship. We propose a rectification module that uses such prior knowledge to refine model generated pseudo labels.
  arXiv  Detail & Related papers  (2022-07-18T18:41:36Z)
• f-Domain-Adversarial Learning: Theory and Algorithms [82.97698406515667]
  Unsupervised domain adaptation is used in many machine learning applications where, during training, a model has access to unlabeled data in the target domain. We derive a novel generalization bound for domain adaptation that exploits a new measure of discrepancy between distributions based on a variational characterization of f-divergences.
  arXiv  Detail & Related papers  (2021-06-21T18:21:09Z)
• Dual-Teacher++: Exploiting Intra-domain and Inter-domain Knowledge with Reliable Transfer for Cardiac Segmentation [69.09432302497116]
  We propose a cutting-edge semi-supervised domain adaptation framework, namely Dual-Teacher++. We design novel dual teacher models, including an inter-domain teacher model to explore cross-modality priors from source domain (e.g., MR) and an intra-domain teacher model to investigate the knowledge beneath unlabeled target domain. In this way, the student model can obtain reliable dual-domain knowledge and yield improved performance on target domain data.
  arXiv  Detail & Related papers  (2021-01-07T05:17:38Z)
• A survey on domain adaptation theory: learning bounds and theoretical guarantees [17.71634393160982]
  The main objective of this survey is to provide an overview of the state-of-the-art theoretical results in a specific, and arguably the most popular, sub-field of transfer learning. In this sub-field, the data distribution is assumed to change across the training and the test data, while the learning task remains the same. We provide a first up-to-date description of existing results related to domain adaptation problem.
  arXiv  Detail & Related papers  (2020-04-24T16:11:03Z)
• Domain Adaption for Knowledge Tracing [65.86619804954283]
  We propose a novel adaptable framework, namely knowledge tracing (AKT) to address the DAKT problem. For the first aspect, we incorporate the educational characteristics (e.g., slip, guess, question texts) based on the deep knowledge tracing (DKT) to obtain a good performed knowledge tracing model. For the second aspect, we propose and adopt three domain adaptation processes. First, we pre-train an auto-encoder to select useful source instances for target model training.
  arXiv  Detail & Related papers  (2020-01-14T15:04:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.