SAICL: Student Modelling with Interaction-level Auxiliary Contrastive Tasks for Knowledge Tracing and Dropout Prediction

• URL: http://arxiv.org/abs/2210.09012v2
• Date: Wed, 19 Oct 2022 12:50:56 GMT
• Title: SAICL: Student Modelling with Interaction-level Auxiliary Contrastive Tasks for Knowledge Tracing and Dropout Prediction
• Authors: Jungbae Park, Jinyoung Kim, Soonwoo Kwon, and Sang Wan Lee
• Abstract summary: This study introduces a novel student modeling framework, SAICL. By combining cross-entropy with contrastive objectives, the proposed SAICL achieved comparable knowledge tracing and dropout prediction performance.
• Score: 15.116940192251029
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Knowledge tracing and dropout prediction are crucial for online education to estimate students' knowledge states or to prevent dropout rates. While traditional systems interacting with students suffered from data sparsity and overfitting, recent sample-level contrastive learning helps to alleviate this issue. One major limitation of sample-level approaches is that they regard students' behavior interaction sequences as a bundle, so they often fail to encode temporal contexts and track their dynamic changes, making it hard to find optimal representations for knowledge tracing and dropout prediction. To apply temporal context within the sequence, this study introduces a novel student modeling framework, SAICL: \textbf{s}tudent modeling with \textbf{a}uxiliary \textbf{i}nteraction-level \textbf{c}ontrastive \textbf{l}earning. In detail, SAICL can utilize both proposed self-supervised/supervised interaction-level contrastive objectives: MilCPC (\textbf{M}ulti-\textbf{I}nteraction-\textbf{L}evel \textbf{C}ontrastive \textbf{P}redictive \textbf{C}oding) and SupCPC (\textbf{Sup}ervised \textbf{C}ontrastive \textbf{P}redictive \textbf{C}oding). While previous sample-level contrastive methods for student modeling are highly dependent on data augmentation methods, the SAICL is free of data augmentation while showing better performance in both self-supervised and supervised settings. By combining cross-entropy with contrastive objectives, the proposed SAICL achieved comparable knowledge tracing and dropout prediction performance with other state-of-art models without compromising inference costs.

Related papers

• Causality-aligned Prompt Learning via Diffusion-based Counterfactual Generation [45.395353088233556]
  We introduce a theoretically grounded $textbfDi$ffusion-based $textbfC$ounterf$textbfa$ctual $textbfp$rompt learning framework.<n>Our method performs excellently across tasks such as image classification, image-text retrieval, and visual question answering, with particularly strong advantages in unseen categories.
  arXiv  Detail & Related papers  (2025-07-26T09:27:52Z)
• Robust Molecular Property Prediction via Densifying Scarce Labeled Data [51.55434084913129]
  In drug discovery, compounds most critical for advancing research often lie beyond the training set.<n>We propose a novel meta-learning-based approach that leverages unlabeled data to interpolate between in-distribution (ID) and out-of-distribution (OOD) data.<n>We demonstrate significant performance gains on challenging real-world datasets.
  arXiv  Detail & Related papers  (2025-06-13T15:27:40Z)
• A theoretical framework for self-supervised contrastive learning for continuous dependent data [86.50780641055258]
  Self-supervised learning (SSL) has emerged as a powerful approach to learning representations, particularly in the field of computer vision.<n>We propose a novel theoretical framework for contrastive SSL tailored to emphsemantic independence between samples.<n>Specifically, we outperform TS2Vec on the standard UEA and UCR benchmarks, with accuracy improvements of $4.17$% and $2.08$%, respectively.
  arXiv  Detail & Related papers  (2025-06-11T14:23:47Z)
• Mitigating Spurious Correlations in LLMs via Causality-Aware Post-Training [57.03005244917803]
  Large language models (LLMs) often fail on out-of-distribution (OOD) samples due to spurious correlations acquired during pre-training.<n>Here, we aim to mitigate such spurious correlations through causality-aware post-training (CAPT)<n> Experiments on the formal causal inference benchmark CLadder and the logical reasoning dataset PrOntoQA show that 3B-scale language models fine-tuned with CAPT can outperform both traditional SFT and larger LLMs on in-distribution (ID) and OOD tasks.
  arXiv  Detail & Related papers  (2025-06-11T06:30:28Z)
• Dynamic Relation Inference via Verb Embeddings [2.8436327410529483]
  We offer insights and practical methods to advance the field of relation inference from images. We propose Dynamic Relation Inference via Verb Embeddings (DRIVE), which augments the COCO dataset, fine-tunes CLIP with hard negatives subject-relation-object triples and corresponding images, and introduces a novel loss function to improve relation detection.
  arXiv  Detail & Related papers  (2025-03-17T10:24:27Z)
• Unpacking the Resilience of SNLI Contradiction Examples to Attacks [0.38366697175402226]
  We apply the Universal Adversarial Attack to examine the model's vulnerabilities. Our analysis revealed substantial drops in accuracy for the entailment and neutral classes. Fine-tuning the model on an augmented dataset with adversarial examples restored its performance to near-baseline levels.
  arXiv  Detail & Related papers  (2024-12-15T12:47:28Z)
• Towards Robust Recommendation via Decision Boundary-aware Graph Contrastive Learning [25.514007761856632]
  graph contrastive learning (GCL) has received increasing attention in recommender systems due to its effectiveness in reducing bias caused by data sparsity. We argue that these methods struggle to balance between semantic invariance and view hardness across the dynamic training process. We propose a novel GCL-based recommendation framework RGCL, which effectively maintains the semantic invariance of contrastive pairs and dynamically adapts as the model capability evolves.
  arXiv  Detail & Related papers  (2024-07-14T13:03:35Z)
• CELA: Cost-Efficient Language Model Alignment for CTR Prediction [71.85120354973073]
  Click-Through Rate (CTR) prediction holds a paramount position in recommender systems. Recent efforts have sought to mitigate these challenges by integrating Pre-trained Language Models (PLMs) We propose textbfCost-textbfEfficient textbfLanguage Model textbfAlignment (textbfCELA) for CTR prediction.
  arXiv  Detail & Related papers  (2024-05-17T07:43:25Z)
• Noisy Correspondence Learning with Self-Reinforcing Errors Mitigation [63.180725016463974]
  Cross-modal retrieval relies on well-matched large-scale datasets that are laborious in practice. We introduce a novel noisy correspondence learning framework, namely textbfSelf-textbfReinforcing textbfErrors textbfMitigation (SREM)
  arXiv  Detail & Related papers  (2023-12-27T09:03:43Z)
• Disentangled Representation Learning with Transmitted Information Bottleneck [57.22757813140418]
  We present textbfDisTIB (textbfTransmitted textbfInformation textbfBottleneck for textbfDisd representation learning), a novel objective that navigates the balance between information compression and preservation.
  arXiv  Detail & Related papers  (2023-11-03T03:18:40Z)
• IDRNet: Intervention-Driven Relation Network for Semantic Segmentation [34.09179171102469]
  Co-occurrent visual patterns suggest that pixel relation modeling facilitates dense prediction tasks. Despite the impressive results, existing paradigms often suffer from inadequate or ineffective contextual information aggregation. We propose a novel textbfIntervention-textbfDriven textbfRelation textbfNetwork.
  arXiv  Detail & Related papers  (2023-10-16T18:37:33Z)
• STS-CCL: Spatial-Temporal Synchronous Contextual Contrastive Learning for Urban Traffic Forecasting [4.947443433688782]
  This work employs the advanced contrastive learning and proposes a novel Spatial-Temporalous Contextual Contrastive Learning (STS-CCL) model. Experiments and evaluations demonstrate that building a predictor upon STS-CCL contrastive learning model gains superior performance than existing traffic forecasting benchmarks.
  arXiv  Detail & Related papers  (2023-07-05T03:47:28Z)
• Implicit Counterfactual Data Augmentation for Robust Learning [24.795542869249154]
  This study proposes an Implicit Counterfactual Data Augmentation method to remove spurious correlations and make stable predictions.<n>Experiments have been conducted across various biased learning scenarios covering both image and text datasets.
  arXiv  Detail & Related papers  (2023-04-26T10:36:40Z)
• Mitigating Data Sparsity for Short Text Topic Modeling by Topic-Semantic Contrastive Learning [19.7066703371736]
  We propose a novel short text topic modeling framework, Topic-Semantic Contrastive Topic Model (TSCTM) Our TSCTM outperforms state-of-the-art baselines regardless of the data augmentation availability, producing high-quality topics and topic distributions.
  arXiv  Detail & Related papers  (2022-11-23T11:33:43Z)
• Adversarial Training with Complementary Labels: On the Benefit of Gradually Informative Attacks [119.38992029332883]
  Adversarial training with imperfect supervision is significant but receives limited attention. We propose a new learning strategy using gradually informative attacks. Experiments are conducted to demonstrate the effectiveness of our method on a range of benchmarked datasets.
  arXiv  Detail & Related papers  (2022-11-01T04:26:45Z)
• SAIS: Supervising and Augmenting Intermediate Steps for Document-Level Relation Extraction [51.27558374091491]
  We propose to explicitly teach the model to capture relevant contexts and entity types by supervising and augmenting intermediate steps (SAIS) for relation extraction. Based on a broad spectrum of carefully designed tasks, our proposed SAIS method not only extracts relations of better quality due to more effective supervision, but also retrieves the corresponding supporting evidence more accurately.
  arXiv  Detail & Related papers  (2021-09-24T17:37:35Z)
• Dense Contrastive Visual-Linguistic Pretraining [53.61233531733243]
  Several multimodal representation learning approaches have been proposed that jointly represent image and text. These approaches achieve superior performance by capturing high-level semantic information from large-scale multimodal pretraining. We propose unbiased Dense Contrastive Visual-Linguistic Pretraining to replace the region regression and classification with cross-modality region contrastive learning.
  arXiv  Detail & Related papers  (2021-09-24T07:20:13Z)
• Contrastive Self-supervised Sequential Recommendation with Robust Augmentation [101.25762166231904]
  Sequential Recommendationdescribes a set of techniques to model dynamic user behavior in order to predict future interactions in sequential user data. Old and new issues remain, including data-sparsity and noisy data. We propose Contrastive Self-Supervised Learning for sequential Recommendation (CoSeRec)
  arXiv  Detail & Related papers  (2021-08-14T07:15:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.