BERT-Based Approach for Automating Course Articulation Matrix Construction with Explainable AI

• URL: http://arxiv.org/abs/2411.14254v1
• Date: Thu, 21 Nov 2024 16:02:39 GMT
• Title: BERT-Based Approach for Automating Course Articulation Matrix Construction with Explainable AI
• Authors: Natenaile Asmamaw Shiferaw, Simpenzwe Honore Leandre, Aman Sinha, Dillip Rout,
• Abstract summary: Course Outcome (CO) and Program Outcome (PO)/Program-Specific Outcome (PSO) alignment is a crucial task for ensuring curriculum coherence and assessing educational effectiveness. This work demonstrates the potential of utilizing transfer learning with BERT-based models for the automated generation of Course Articulation Matrix (CAM) Our system achieves accuracy, precision, recall, and F1-score values of 98.66%, 98.67%, 98.66%, and 98.66%, respectively.
• Score: 1.4214002697449326
• License:
• Abstract: Course Outcome (CO) and Program Outcome (PO)/Program-Specific Outcome (PSO) alignment is a crucial task for ensuring curriculum coherence and assessing educational effectiveness. The construction of a Course Articulation Matrix (CAM), which quantifies the relationship between COs and POs/PSOs, typically involves assigning numerical values (0, 1, 2, 3) to represent the degree of alignment. In this study, We experiment with four models from the BERT family: BERT Base, DistilBERT, ALBERT, and RoBERTa, and use multiclass classification to assess the alignment between CO and PO/PSO pairs. We first evaluate traditional machine learning classifiers, such as Decision Tree, Random Forest, and XGBoost, and then apply transfer learning to evaluate the performance of the pretrained BERT models. To enhance model interpretability, we apply Explainable AI technique, specifically Local Interpretable Model-agnostic Explanations (LIME), to provide transparency into the decision-making process. Our system achieves accuracy, precision, recall, and F1-score values of 98.66%, 98.67%, 98.66%, and 98.66%, respectively. This work demonstrates the potential of utilizing transfer learning with BERT-based models for the automated generation of CAMs, offering high performance and interpretability in educational outcome assessment.

Related papers

• Language Model Preference Evaluation with Multiple Weak Evaluators [78.53743237977677]
  GED (Preference Graph Ensemble and Denoise) is a novel approach that leverages multiple model-based evaluators to construct preference graphs. We show that GED outperforms baseline methods in model ranking, response selection, and model alignment tasks.
  arXiv  Detail & Related papers  (2024-10-14T01:57:25Z)
• Rubric-based Learner Modelling via Noisy Gates Bayesian Networks for Computational Thinking Skills Assessment [40.06500618820166]
  We develop a learner model for automatic skill assessment from a task-specific competence rubric. We design a network with two layers of gates, one performing disjunctive operations by noisy-OR gates and the other conjunctive operations through logical ANDs. The CT-cube skills assessment framework and the Cross Array Task (CAT) are used to exemplify it and demonstrate its feasibility.
  arXiv  Detail & Related papers  (2024-08-02T12:21:05Z)
• CALICO: Confident Active Learning with Integrated Calibration [11.978551396144532]
  We propose an AL framework that self-calibrates the confidence used for sample selection during the training process. We show improved classification performance compared to a softmax-based classifier with fewer labeled samples.
  arXiv  Detail & Related papers  (2024-07-02T15:05:19Z)
• Self-Training with Pseudo-Label Scorer for Aspect Sentiment Quad Prediction [54.23208041792073]
  Aspect Sentiment Quad Prediction (ASQP) aims to predict all quads (aspect term, aspect category, opinion term, sentiment polarity) for a given review. A key challenge in the ASQP task is the scarcity of labeled data, which limits the performance of existing methods. We propose a self-training framework with a pseudo-label scorer, wherein a scorer assesses the match between reviews and their pseudo-labels.
  arXiv  Detail & Related papers  (2024-06-26T05:30:21Z)
• Blending gradient boosted trees and neural networks for point and probabilistic forecasting of hierarchical time series [0.0]
  We describe a blending methodology of machine learning models that belong to gradient boosted trees and neural networks families. These principles were successfully applied in the recent M5 Competition on both Accuracy and Uncertainty tracks.
  arXiv  Detail & Related papers  (2023-10-19T09:42:02Z)
• FLASK: Fine-grained Language Model Evaluation based on Alignment Skill Sets [69.91340332545094]
  We introduce FLASK, a fine-grained evaluation protocol for both human-based and model-based evaluation. We experimentally observe that the fine-graininess of evaluation is crucial for attaining a holistic view of model performance.
  arXiv  Detail & Related papers  (2023-07-20T14:56:35Z)
• Weakly Supervised Semantic Segmentation via Alternative Self-Dual Teaching [82.71578668091914]
  This paper establishes a compact learning framework that embeds the classification and mask-refinement components into a unified deep model. We propose a novel alternative self-dual teaching (ASDT) mechanism to encourage high-quality knowledge interaction.
  arXiv  Detail & Related papers  (2021-12-17T11:56:56Z)
• An Automated Knowledge Mining and Document Classification System with Multi-model Transfer Learning [1.1852751647387592]
  Service manual documents are crucial to the engineering company as they provide guidelines and knowledge to service engineers. We propose an automated knowledge mining and document classification system with novel multi-model transfer learning approaches.
  arXiv  Detail & Related papers  (2021-06-24T03:03:46Z)
• Can Active Learning Preemptively Mitigate Fairness Issues? [66.84854430781097]
  dataset bias is one of the prevailing causes of unfairness in machine learning. We study whether models trained with uncertainty-based ALs are fairer in their decisions with respect to a protected class. We also explore the interaction of algorithmic fairness methods such as gradient reversal (GRAD) and BALD.
  arXiv  Detail & Related papers  (2021-04-14T14:20:22Z)
• Few-Shot Named Entity Recognition: A Comprehensive Study [92.40991050806544]
  We investigate three schemes to improve the model generalization ability for few-shot settings. We perform empirical comparisons on 10 public NER datasets with various proportions of labeled data. We create new state-of-the-art results on both few-shot and training-free settings.
  arXiv  Detail & Related papers  (2020-12-29T23:43:16Z)
• SE3M: A Model for Software Effort Estimation Using Pre-trained Embedding Models [0.8287206589886881]
  This paper proposes to evaluate the effectiveness of pre-trained embeddings models. Generic pre-trained models for both approaches went through a fine-tuning process. Results were very promising, realizing that pre-trained models can be used to estimate software effort based only on requirements texts.
  arXiv  Detail & Related papers  (2020-06-30T14:15:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.