ReliCD: A Reliable Cognitive Diagnosis Framework with Confidence Awareness

• URL: http://arxiv.org/abs/2401.10749v1
• Date: Fri, 29 Dec 2023 07:30:58 GMT
• Title: ReliCD: A Reliable Cognitive Diagnosis Framework with Confidence Awareness
• Authors: Yunfei Zhang, Chuan Qin, Dazhong Shen, Haiping Ma, Le Zhang, Xingyi Zhang, Hengshu Zhu
• Abstract summary: Existing approaches often suffer from the issue of overconfidence in predicting students' mastery levels. We propose a novel Reliable Cognitive Diagnosis(ReliCD) framework, which can quantify the confidence of the diagnosis feedback.
• Score: 26.60714613122676
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: During the past few decades, cognitive diagnostics modeling has attracted increasing attention in computational education communities, which is capable of quantifying the learning status and knowledge mastery levels of students. Indeed, the recent advances in neural networks have greatly enhanced the performance of traditional cognitive diagnosis models through learning the deep representations of students and exercises. Nevertheless, existing approaches often suffer from the issue of overconfidence in predicting students' mastery levels, which is primarily caused by the unavoidable noise and sparsity in realistic student-exercise interaction data, severely hindering the educational application of diagnostic feedback. To address this, in this paper, we propose a novel Reliable Cognitive Diagnosis(ReliCD) framework, which can quantify the confidence of the diagnosis feedback and is flexible for different cognitive diagnostic functions. Specifically, we first propose a Bayesian method to explicitly estimate the state uncertainty of different knowledge concepts for students, which enables the confidence quantification of diagnostic feedback. In particular, to account for potential differences, we suggest modeling individual prior distributions for the latent variables of different ability concepts using a pre-trained model. Additionally, we introduce a logical hypothesis for ranking confidence levels. Along this line, we design a novel calibration loss to optimize the confidence parameters by modeling the process of student performance prediction. Finally, extensive experiments on four real-world datasets clearly demonstrate the effectiveness of our ReliCD framework.

Related papers

• A Survey of Models for Cognitive Diagnosis: New Developments and Future Directions [66.40362209055023]
  This paper aims to provide a survey of current models for cognitive diagnosis, with more attention on new developments using machine learning-based methods. By comparing the model structures, parameter estimation algorithms, model evaluation methods and applications, we provide a relatively comprehensive review of the recent trends in cognitive diagnosis models.
  arXiv  Detail & Related papers  (2024-07-07T18:02:00Z)
• Confidence-aware multi-modality learning for eye disease screening [58.861421804458395]
  We propose a novel multi-modality evidential fusion pipeline for eye disease screening. It provides a measure of confidence for each modality and elegantly integrates the multi-modality information. Experimental results on both public and internal datasets demonstrate that our model excels in robustness.
  arXiv  Detail & Related papers  (2024-05-28T13:27:30Z)
• Decoding Decision Reasoning: A Counterfactual-Powered Model for Knowledge Discovery [6.1521675665532545]
  In medical imaging, discerning the rationale behind an AI model's predictions is crucial for evaluating its reliability. We propose an explainable model that is equipped with both decision reasoning and feature identification capabilities. By implementing our method, we can efficiently identify and visualise class-specific features leveraged by the data-driven model.
  arXiv  Detail & Related papers  (2024-05-23T19:00:38Z)
• Uncertainty-aware Evidential Fusion-based Learning for Semi-supervised Medical Image Segmentation [0.0]
  This paper integrates the evidential predictive results in the cross-region of mixed and original samples to reallocate the confidence degree and uncertainty measure of each voxel. We design a voxel-level learning strategy by introducing information entropy to combine with the fused uncertainty measure to estimate voxel prediction more precisely. The experimental results on LA, Pancreas-CT, ACDC and TBAD datasets demonstrate the superior performance of our proposed method in comparison with the existing state of the arts.
  arXiv  Detail & Related papers  (2024-04-09T09:58:10Z)
• Unified Uncertainty Estimation for Cognitive Diagnosis Models [70.46998436898205]
  We propose a unified uncertainty estimation approach for a wide range of cognitive diagnosis models. We decompose the uncertainty of diagnostic parameters into data aspect and model aspect. Our method is effective and can provide useful insights into the uncertainty of cognitive diagnosis.
  arXiv  Detail & Related papers  (2024-03-09T13:48:20Z)
• A Foundational Framework and Methodology for Personalized Early and Timely Diagnosis [84.6348989654916]
  We propose the first foundational framework for early and timely diagnosis. It builds on decision-theoretic approaches to outline the diagnosis process. It integrates machine learning and statistical methodology for estimating the optimal personalized diagnostic path.
  arXiv  Detail & Related papers  (2023-11-26T14:42:31Z)
• Towards the Identifiability and Explainability for Personalized Learner Modeling: An Inductive Paradigm [36.60917255464867]
  We propose an identifiable cognitive diagnosis framework (ID-CDF) based on a novel response-proficiency-response paradigm inspired by encoder-decoder models. We show that ID-CDF can effectively address the problems without loss of diagnosis preciseness.
  arXiv  Detail & Related papers  (2023-09-01T07:18:02Z)
• UIILD: A Unified Interpretable Intelligent Learning Diagnosis Framework for Intelligent Tutoring Systems [8.354034992258482]
  The proposed unified interpretable intelligent learning diagnosis (UIILD) framework benefits from the powerful representation learning ability of deep learning and the interpretability of psychometrics. Within the proposed framework, this paper presents a two-channel learning diagnosis mechanism LDM-ID as well as a three-channel learning diagnosis mechanism LDM-HMI.
  arXiv  Detail & Related papers  (2022-07-07T07:04:22Z)
• Estimating and Improving Fairness with Adversarial Learning [65.99330614802388]
  We propose an adversarial multi-task training strategy to simultaneously mitigate and detect bias in the deep learning-based medical image analysis system. Specifically, we propose to add a discrimination module against bias and a critical module that predicts unfairness within the base classification model. We evaluate our framework on a large-scale public-available skin lesion dataset.
  arXiv  Detail & Related papers  (2021-03-07T03:10:32Z)
• Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integration [55.88616573143478]
  Outcome prediction from clinical text can prevent doctors from overlooking possible risks. Diagnoses at discharge, procedures performed, in-hospital mortality and length-of-stay prediction are four common outcome prediction targets. We propose clinical outcome pre-training to integrate knowledge about patient outcomes from multiple public sources.
  arXiv  Detail & Related papers  (2021-02-08T10:26:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.