Trusting the Explainers: Teacher Validation of Explainable Artificial Intelligence for Course Design

• URL: http://arxiv.org/abs/2212.08955v2
• Date: Mon, 26 Dec 2022 02:32:58 GMT
• Title: Trusting the Explainers: Teacher Validation of Explainable Artificial Intelligence for Course Design
• Authors: Vinitra Swamy, Sijia Du, Mirko Marras, Tanja K\"aser
• Abstract summary: This work focuses on the context of online and blended learning and the use case of student success prediction models. We use a pairwise study design, enabling us to investigate controlled differences between pairs of courses. We quantitatively compare the distances between the explanations across courses and methods. We then validate the explanations of LIME and SHAP with 26 semi-structured interviews of university-level educators.
• Score: 5.725477071353353
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep learning models for learning analytics have become increasingly popular over the last few years; however, these approaches are still not widely adopted in real-world settings, likely due to a lack of trust and transparency. In this paper, we tackle this issue by implementing explainable AI methods for black-box neural networks. This work focuses on the context of online and blended learning and the use case of student success prediction models. We use a pairwise study design, enabling us to investigate controlled differences between pairs of courses. Our analyses cover five course pairs that differ in one educationally relevant aspect and two popular instance-based explainable AI methods (LIME and SHAP). We quantitatively compare the distances between the explanations across courses and methods. We then validate the explanations of LIME and SHAP with 26 semi-structured interviews of university-level educators regarding which features they believe contribute most to student success, which explanations they trust most, and how they could transform these insights into actionable course design decisions. Our results show that quantitatively, explainers significantly disagree with each other about what is important, and qualitatively, experts themselves do not agree on which explanations are most trustworthy. All code, extended results, and the interview protocol are provided at https://github.com/epfl-ml4ed/trusting-explainers.

Related papers

• Fool Me Once? Contrasting Textual and Visual Explanations in a Clinical Decision-Support Setting [43.110187812734864]
  We evaluate three types of explanations: visual explanations (saliency maps), natural language explanations, and a combination of both modalities. We find that text-based explanations lead to significant over-reliance, which is alleviated by combining them with saliency maps. We also observe that the quality of explanations, that is, how much factually correct information they entail, and how much this aligns with AI correctness, significantly impacts the usefulness of the different explanation types.
  arXiv  Detail & Related papers  (2024-10-16T06:43:02Z)
• Solving the enigma: Deriving optimal explanations of deep networks [3.9584068556746246]
  We propose a novel framework designed to enhance the explainability of deep networks. Our framework integrates various explanations from established XAI methods and employs a non-explanation to construct an optimal explanation. Our results suggest that optimal explanations based on specific criteria are derivable.
  arXiv  Detail & Related papers  (2024-05-16T11:49:08Z)
• Context-faithful Prompting for Large Language Models [51.194410884263135]
  Large language models (LLMs) encode parametric knowledge about world facts. Their reliance on parametric knowledge may cause them to overlook contextual cues, leading to incorrect predictions in context-sensitive NLP tasks. We assess and enhance LLMs' contextual faithfulness in two aspects: knowledge conflict and prediction with abstention.
  arXiv  Detail & Related papers  (2023-03-20T17:54:58Z)
• Explaining Model Confidence Using Counterfactuals [4.385390451313721]
  Displaying confidence scores in human-AI interaction has been shown to help build trust between humans and AI systems. Most existing research uses only the confidence score as a form of communication. We show that counterfactual explanations of confidence scores help study participants to better understand and better trust a machine learning model's prediction.
  arXiv  Detail & Related papers  (2023-03-10T06:22:13Z)
• See, Think, Confirm: Interactive Prompting Between Vision and Language Models for Knowledge-based Visual Reasoning [60.43585179885355]
  We propose a novel framework named Interactive Prompting Visual Reasoner (IPVR) for few-shot knowledge-based visual reasoning. IPVR contains three stages, see, think and confirm. We conduct experiments on a range of knowledge-based visual reasoning datasets.
  arXiv  Detail & Related papers  (2023-01-12T18:59:50Z)
• Evaluating the Explainers: Black-Box Explainable Machine Learning for Student Success Prediction in MOOCs [5.241055914181294]
  We implement five state-of-the-art methodologies for explaining black-box machine learning models. We examine the strengths of each approach on the downstream task of student performance prediction. Our results come to the concerning conclusion that the choice of explainer is an important decision.
  arXiv  Detail & Related papers  (2022-07-01T17:09:17Z)
• Learning to Scaffold: Optimizing Model Explanations for Teaching [74.25464914078826]
  We train models on three natural language processing and computer vision tasks. We find that students trained with explanations extracted with our framework are able to simulate the teacher significantly more effectively than ones produced with previous methods.
  arXiv  Detail & Related papers  (2022-04-22T16:43:39Z)
• REX: Reasoning-aware and Grounded Explanation [30.392986232906107]
  We develop a new type of multi-modal explanations that explain the decisions by traversing the reasoning process and grounding keywords in the images. Second, we identify the critical need to tightly couple important components across the visual and textual modalities for explaining the decisions. Third, we propose a novel explanation generation method that explicitly models the pairwise correspondence between words and regions of interest.
  arXiv  Detail & Related papers  (2022-03-11T17:28:42Z)
• The Who in XAI: How AI Background Shapes Perceptions of AI Explanations [61.49776160925216]
  We conduct a mixed-methods study of how two different groups--people with and without AI background--perceive different types of AI explanations. We find that (1) both groups showed unwarranted faith in numbers for different reasons and (2) each group found value in different explanations beyond their intended design.
  arXiv  Detail & Related papers  (2021-07-28T17:32:04Z)
• Exploring Bayesian Deep Learning for Urgent Instructor Intervention Need in MOOC Forums [58.221459787471254]
  Massive Open Online Courses (MOOCs) have become a popular choice for e-learning thanks to their great flexibility. Due to large numbers of learners and their diverse backgrounds, it is taxing to offer real-time support. With the large volume of posts and high workloads for MOOC instructors, it is unlikely that the instructors can identify all learners requiring intervention. This paper explores for the first time Bayesian deep learning on learner-based text posts with two methods: Monte Carlo Dropout and Variational Inference.
  arXiv  Detail & Related papers  (2021-04-26T15:12:13Z)
• Explainable Active Learning (XAL): An Empirical Study of How Local Explanations Impact Annotator Experience [76.9910678786031]
  We propose a novel paradigm of explainable active learning (XAL), by introducing techniques from the recently surging field of explainable AI (XAI) into an Active Learning setting. Our study shows benefits of AI explanation as interfaces for machine teaching--supporting trust calibration and enabling rich forms of teaching feedback, and potential drawbacks--anchoring effect with the model judgment and cognitive workload.
  arXiv  Detail & Related papers  (2020-01-24T22:52:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.