HIVE: Evaluating the Human Interpretability of Visual Explanations

• URL: http://arxiv.org/abs/2112.03184v1
• Date: Mon, 6 Dec 2021 17:30:47 GMT
• Title: HIVE: Evaluating the Human Interpretability of Visual Explanations
• Authors: Sunnie S. Y. Kim and Nicole Meister and Vikram V. Ramaswamy and Ruth Fong and Olga Russakovsky
• Abstract summary: We propose a novel human evaluation framework HIVE (Human Interpretability of Visual Explanations) for diverse interpretability methods in computer vision. Our results suggest that explanations (regardless of if they are actually correct) engender human trust, yet are not distinct enough for users to distinguish between correct and incorrect predictions.
• Score: 20.060507122989645
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As machine learning is increasingly applied to high-impact, high-risk domains, there have been a number of new methods aimed at making AI models more human interpretable. Despite the recent growth of interpretability work, there is a lack of systematic evaluation of proposed techniques. In this work, we propose a novel human evaluation framework HIVE (Human Interpretability of Visual Explanations) for diverse interpretability methods in computer vision; to the best of our knowledge, this is the first work of its kind. We argue that human studies should be the gold standard in properly evaluating how interpretable a method is to human users. While human studies are often avoided due to challenges associated with cost, study design, and cross-method comparison, we describe how our framework mitigates these issues and conduct IRB-approved studies of four methods that represent the diversity of interpretability works: GradCAM, BagNet, ProtoPNet, and ProtoTree. Our results suggest that explanations (regardless of if they are actually correct) engender human trust, yet are not distinct enough for users to distinguish between correct and incorrect predictions. Lastly, we also open-source our framework to enable future studies and to encourage more human-centered approaches to interpretability.

Related papers

• Human-Modeling in Sequential Decision-Making: An Analysis through the Lens of Human-Aware AI [20.21053807133341]
  We try to provide an account of what constitutes a human-aware AI system. We see that human-aware AI is a design oriented paradigm, one that focuses on the need for modeling the humans it may interact with.
  arXiv  Detail & Related papers  (2024-05-13T14:17:52Z)
• Evaluating the Utility of Model Explanations for Model Development [54.23538543168767]
  We evaluate whether explanations can improve human decision-making in practical scenarios of machine learning model development. To our surprise, we did not find evidence of significant improvement on tasks when users were provided with any of the saliency maps. These findings suggest caution regarding the usefulness and potential for misunderstanding in saliency-based explanations.
  arXiv  Detail & Related papers  (2023-12-10T23:13:23Z)
• Interpreting Pretrained Language Models via Concept Bottlenecks [55.47515772358389]
  Pretrained language models (PLMs) have made significant strides in various natural language processing tasks. The lack of interpretability due to their black-box'' nature poses challenges for responsible implementation. We propose a novel approach to interpreting PLMs by employing high-level, meaningful concepts that are easily understandable for humans.
  arXiv  Detail & Related papers  (2023-11-08T20:41:18Z)
• Towards Fair and Explainable AI using a Human-Centered AI Approach [5.888646114353372]
  We present 5 research projects that aim to enhance explainability and fairness in classification systems and word embeddings. The first project explores the utility/downsides of introducing local model explanations as interfaces for machine teachers. The second project presents D-BIAS, a causality-based human-in-the-loop visual tool for identifying and mitigating social biases in datasets. The third project presents WordBias, a visual interactive tool that helps audit pre-trained static word embeddings for biases against groups. The fourth project presents DramatVis Personae, a visual analytics tool that helps identify social
  arXiv  Detail & Related papers  (2023-06-12T21:08:55Z)
• Human Uncertainty in Concept-Based AI Systems [37.82747673914624]
  We study human uncertainty in the context of concept-based AI systems. We show that training with uncertain concept labels may help mitigate weaknesses in concept-based systems.
  arXiv  Detail & Related papers  (2023-03-22T19:17:57Z)
• Empirical Estimates on Hand Manipulation are Recoverable: A Step Towards Individualized and Explainable Robotic Support in Everyday Activities [80.37857025201036]
  Key challenge for robotic systems is to figure out the behavior of another agent. Processing correct inferences is especially challenging when (confounding) factors are not controlled experimentally. We propose equipping robots with the necessary tools to conduct observational studies on people.
  arXiv  Detail & Related papers  (2022-01-27T22:15:56Z)
• Quality Metrics for Transparent Machine Learning With and Without Humans In the Loop Are Not Correlated [0.0]
  We investigate the quality of interpretable computer vision algorithms using techniques from psychophysics. Our results demonstrate that psychophysical experiments allow for robust quality assessment of transparency in machine learning.
  arXiv  Detail & Related papers  (2021-07-01T12:30:51Z)
• Interpretable Deep Learning: Interpretations, Interpretability, Trustworthiness, and Beyond [49.93153180169685]
  We introduce and clarify two basic concepts-interpretations and interpretability-that people usually get confused. We elaborate the design of several recent interpretation algorithms, from different perspectives, through proposing a new taxonomy. We summarize the existing work in evaluating models' interpretability using "trustworthy" interpretation algorithms.
  arXiv  Detail & Related papers  (2021-03-19T08:40:30Z)
• Evaluating the Interpretability of Generative Models by Interactive Reconstruction [30.441247705313575]
  We introduce a task to quantify the human-interpretability of generative model representations. We find performance on this task much more reliably differentiates entangled and disentangled models than baseline approaches.
  arXiv  Detail & Related papers  (2021-02-02T02:38:14Z)
• An Uncertainty-based Human-in-the-loop System for Industrial Tool Wear Analysis [68.8204255655161]
  We show that uncertainty measures based on Monte-Carlo dropout in the context of a human-in-the-loop system increase the system's transparency and performance. A simulation study demonstrates that the uncertainty-based human-in-the-loop system increases performance for different levels of human involvement.
  arXiv  Detail & Related papers  (2020-07-14T15:47:37Z)
• Human Trajectory Forecasting in Crowds: A Deep Learning Perspective [89.4600982169]
  We present an in-depth analysis of existing deep learning-based methods for modelling social interactions. We propose two knowledge-based data-driven methods to effectively capture these social interactions. We develop a large scale interaction-centric benchmark TrajNet++, a significant yet missing component in the field of human trajectory forecasting.
  arXiv  Detail & Related papers  (2020-07-07T17:19:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.