Quality Metrics for Transparent Machine Learning With and Without Humans In the Loop Are Not Correlated

• URL: http://arxiv.org/abs/2107.02033v1
• Date: Thu, 1 Jul 2021 12:30:51 GMT
• Title: Quality Metrics for Transparent Machine Learning With and Without Humans In the Loop Are Not Correlated
• Authors: Felix Biessmann and Dionysius Refiano
• Abstract summary: We investigate the quality of interpretable computer vision algorithms using techniques from psychophysics. Our results demonstrate that psychophysical experiments allow for robust quality assessment of transparency in machine learning.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: The field explainable artificial intelligence (XAI) has brought about an arsenal of methods to render Machine Learning (ML) predictions more interpretable. But how useful explanations provided by transparent ML methods are for humans remains difficult to assess. Here we investigate the quality of interpretable computer vision algorithms using techniques from psychophysics. In crowdsourced annotation tasks we study the impact of different interpretability approaches on annotation accuracy and task time. We compare these quality metrics with classical XAI, automated quality metrics. Our results demonstrate that psychophysical experiments allow for robust quality assessment of transparency in machine learning. Interestingly the quality metrics computed without humans in the loop did not provide a consistent ranking of interpretability methods nor were they representative for how useful an explanation was for humans. These findings highlight the potential of methods from classical psychophysics for modern machine learning applications. We hope that our results provide convincing arguments for evaluating interpretability in its natural habitat, human-ML interaction, if the goal is to obtain an authentic assessment of interpretability.

Related papers

• A Multimodal Automated Interpretability Agent [63.8551718480664]
  MAIA is a system that uses neural models to automate neural model understanding tasks. We first characterize MAIA's ability to describe (neuron-level) features in learned representations of images. We then show that MAIA can aid in two additional interpretability tasks: reducing sensitivity to spurious features, and automatically identifying inputs likely to be mis-classified.
  arXiv  Detail & Related papers  (2024-04-22T17:55:11Z)
• RLIF: Interactive Imitation Learning as Reinforcement Learning [56.997263135104504]
  We show how off-policy reinforcement learning can enable improved performance under assumptions that are similar but potentially even more practical than those of interactive imitation learning. Our proposed method uses reinforcement learning with user intervention signals themselves as rewards. This relaxes the assumption that intervening experts in interactive imitation learning should be near-optimal and enables the algorithm to learn behaviors that improve over the potential suboptimal human expert.
  arXiv  Detail & Related papers  (2023-11-21T21:05:21Z)
• Interpreting Neural Policies with Disentangled Tree Representations [58.769048492254555]
  We study interpretability of compact neural policies through the lens of disentangled representation. We leverage decision trees to obtain factors of variation for disentanglement in robot learning. We introduce interpretability metrics that measure disentanglement of learned neural dynamics.
  arXiv  Detail & Related papers  (2022-10-13T01:10:41Z)
• Detecting Shortcut Learning for Fair Medical AI using Shortcut Testing [62.9062883851246]
  Machine learning holds great promise for improving healthcare, but it is critical to ensure that its use will not propagate or amplify health disparities. One potential driver of algorithmic unfairness, shortcut learning, arises when ML models base predictions on improper correlations in the training data. Using multi-task learning, we propose the first method to assess and mitigate shortcut learning as a part of the fairness assessment of clinical ML systems.
  arXiv  Detail & Related papers  (2022-07-21T09:35:38Z)
• Process Knowledge-infused Learning for Suicidality Assessment on Social Media [14.362199192484006]
  Current methods rely on the traditional pipeline of predicting labels from data. Post hoc explanations on the data to label prediction using explainable AI (XAI) models leave much to be desired to the end-users. PK-iL utilizes a structured process knowledge that explicitly explains the underlying prediction process that makes sense to end-users.
  arXiv  Detail & Related papers  (2022-04-26T19:43:41Z)
• HIVE: Evaluating the Human Interpretability of Visual Explanations [20.060507122989645]
  We propose a novel human evaluation framework HIVE (Human Interpretability of Visual Explanations) for diverse interpretability methods in computer vision. Our results suggest that explanations (regardless of if they are actually correct) engender human trust, yet are not distinct enough for users to distinguish between correct and incorrect predictions.
  arXiv  Detail & Related papers  (2021-12-06T17:30:47Z)
• Human-in-the-Loop Disinformation Detection: Stance, Sentiment, or Something Else? [93.91375268580806]
  Both politics and pandemics have recently provided ample motivation for the development of machine learning-enabled disinformation (a.k.a. fake news) detection algorithms. Existing literature has focused primarily on the fully-automated case, but the resulting techniques cannot reliably detect disinformation on the varied topics, sources, and time scales required for military applications. By leveraging an already-available analyst as a human-in-the-loop, canonical machine learning techniques of sentiment analysis, aspect-based sentiment analysis, and stance detection become plausible methods to use for a partially-automated disinformation detection system.
  arXiv  Detail & Related papers  (2021-11-09T13:30:34Z)
• What Matters in Learning from Offline Human Demonstrations for Robot Manipulation [64.43440450794495]
  We conduct an extensive study of six offline learning algorithms for robot manipulation. Our study analyzes the most critical challenges when learning from offline human data. We highlight opportunities for learning from human datasets.
  arXiv  Detail & Related papers  (2021-08-06T20:48:30Z)
• A Turing Test for Transparency [0.0]
  A central goal of explainable artificial intelligence (XAI) is to improve the trust relationship in human-AI interaction. Recent empirical evidence shows that explanations can have the opposite effect. This effect challenges the very goal of XAI and implies that responsible usage of transparent AI methods has to consider the ability of humans to distinguish machine generated from human explanations.
  arXiv  Detail & Related papers  (2021-06-21T20:09:40Z)
• A Comparative Approach to Explainable Artificial Intelligence Methods in Application to High-Dimensional Electronic Health Records: Examining the Usability of XAI [0.0]
  XAI aims to produce a demonstrative factor of trust, which for human subjects is achieved through communicative means. The ideology behind trusting a machine to tend towards the livelihood of a human poses an ethical conundrum. XAI methods produce visualization of the feature contribution towards a given models output on both a local and global level.
  arXiv  Detail & Related papers  (2021-03-08T18:15:52Z)
• Sentence Level Human Translation Quality Estimation with Attention-based Neural Networks [0.30458514384586394]
  This paper explores the use of Deep Learning methods for automatic estimation of quality of human translations. Empirical results on a large human annotated dataset show that the neural model outperforms feature-based methods significantly.
  arXiv  Detail & Related papers  (2020-03-13T16:57:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.