Saliency Cards: A Framework to Characterize and Compare Saliency Methods

• URL: http://arxiv.org/abs/2206.02958v2
• Date: Tue, 30 May 2023 20:17:18 GMT
• Title: Saliency Cards: A Framework to Characterize and Compare Saliency Methods
• Authors: Angie Boggust, Harini Suresh, Hendrik Strobelt, John V. Guttag, Arvind Satyanarayan
• Abstract summary: Saliency methods calculate how important each input feature is to a model's output. Existing approaches assume universal desiderata for saliency methods that do not account for diverse user needs. We introduce saliency cards: structured documentation of how saliency methods operate and their performance.
• Score: 34.38335172204263
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Saliency methods are a common class of machine learning interpretability techniques that calculate how important each input feature is to a model's output. We find that, with the rapid pace of development, users struggle to stay informed of the strengths and limitations of new methods and, thus, choose methods for unprincipled reasons (e.g., popularity). Moreover, despite a corresponding rise in evaluation metrics, existing approaches assume universal desiderata for saliency methods (e.g., faithfulness) that do not account for diverse user needs. In response, we introduce saliency cards: structured documentation of how saliency methods operate and their performance across a battery of evaluative metrics. Through a review of 25 saliency method papers and 33 method evaluations, we identify 10 attributes that users should account for when choosing a method. We group these attributes into three categories that span the process of computing and interpreting saliency: methodology, or how the saliency is calculated; sensitivity, or the relationship between the saliency and the underlying model and data; and, perceptibility, or how an end user ultimately interprets the result. By collating this information, saliency cards allow users to more holistically assess and compare the implications of different methods. Through nine semi-structured interviews with users from various backgrounds, including researchers, radiologists, and computational biologists, we find that saliency cards provide a detailed vocabulary for discussing individual methods and allow for a more systematic selection of task-appropriate methods. Moreover, with saliency cards, we are able to analyze the research landscape in a more structured fashion to identify opportunities for new methods and evaluation metrics for unmet user needs.

Related papers

• Identifying Reasons for Bias: An Argumentation-Based Approach [2.9465623430708905]
  We propose a novel model-agnostic argumentation-based method to determine why an individual is classified differently in comparison to similar individuals. We evaluate our method on two datasets commonly used in the fairness literature and illustrate its effectiveness in the identification of bias.
  arXiv  Detail & Related papers  (2023-10-25T09:47:15Z)
• Better Understanding Differences in Attribution Methods via Systematic Evaluations [57.35035463793008]
  Post-hoc attribution methods have been proposed to identify image regions most influential to the models' decisions. We propose three novel evaluation schemes to more reliably measure the faithfulness of those methods. We use these evaluation schemes to study strengths and shortcomings of some widely used attribution methods over a wide range of models.
  arXiv  Detail & Related papers  (2023-03-21T14:24:58Z)
• Reinforced Labels: Multi-Agent Deep Reinforcement Learning for Point-Feature Label Placement [0.0]
  We introduce Reinforcement Learning (RL) to label placement, a complex task in data visualization. Our novel point-feature label placement method utilizes Multi-Agent Deep Reinforcement Learning to learn the label placement strategy. Our results show that the strategy trained by our method significantly outperforms the random strategy of an untrained agent.
  arXiv  Detail & Related papers  (2023-03-02T16:18:00Z)
• Using Active Learning Methods to Strategically Select Essays for Automated Scoring [0.0]
  The purpose of this study is to describe and evaluate three active learning methods. The three active learning methods are the uncertainty-based, the topological-based, and the hybrid method. All three methods produced strong results, with the topological-based method producing the most efficient classification.
  arXiv  Detail & Related papers  (2023-01-02T12:46:10Z)
• Towards Better Understanding Attribution Methods [77.1487219861185]
  Post-hoc attribution methods have been proposed to identify image regions most influential to the models' decisions. We propose three novel evaluation schemes to more reliably measure the faithfulness of those methods. We also propose a post-processing smoothing step that significantly improves the performance of some attribution methods.
  arXiv  Detail & Related papers  (2022-05-20T20:50:17Z)
• Human-in-the-Loop Disinformation Detection: Stance, Sentiment, or Something Else? [93.91375268580806]
  Both politics and pandemics have recently provided ample motivation for the development of machine learning-enabled disinformation (a.k.a. fake news) detection algorithms. Existing literature has focused primarily on the fully-automated case, but the resulting techniques cannot reliably detect disinformation on the varied topics, sources, and time scales required for military applications. By leveraging an already-available analyst as a human-in-the-loop, canonical machine learning techniques of sentiment analysis, aspect-based sentiment analysis, and stance detection become plausible methods to use for a partially-automated disinformation detection system.
  arXiv  Detail & Related papers  (2021-11-09T13:30:34Z)
• Towards Improved and Interpretable Deep Metric Learning via Attentive Grouping [103.71992720794421]
  Grouping has been commonly used in deep metric learning for computing diverse features. We propose an improved and interpretable grouping method to be integrated flexibly with any metric learning framework.
  arXiv  Detail & Related papers  (2020-11-17T19:08:24Z)
• There and Back Again: Revisiting Backpropagation Saliency Methods [87.40330595283969]
  Saliency methods seek to explain the predictions of a model by producing an importance map across each input sample. A popular class of such methods is based on backpropagating a signal and analyzing the resulting gradient. We propose a single framework under which several such methods can be unified.
  arXiv  Detail & Related papers  (2020-04-06T17:58:08Z)
• PONE: A Novel Automatic Evaluation Metric for Open-Domain Generative Dialogue Systems [48.99561874529323]
  There are three kinds of automatic methods to evaluate the open-domain generative dialogue systems. Due to the lack of systematic comparison, it is not clear which kind of metrics are more effective. We propose a novel and feasible learning-based metric that can significantly improve the correlation with human judgments.
  arXiv  Detail & Related papers  (2020-04-06T04:36:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.