What if you said that differently?: How Explanation Formats Affect Human Feedback Efficacy and User Perception

• URL: http://arxiv.org/abs/2311.09558v2
• Date: Tue, 2 Apr 2024 00:59:36 GMT
• Title: What if you said that differently?: How Explanation Formats Affect Human Feedback Efficacy and User Perception
• Authors: Chaitanya Malaviya, Subin Lee, Dan Roth, Mark Yatskar,
• Abstract summary: We analyze the effect of rationales generated by QA models to support their answers. We present users with incorrect answers and corresponding rationales in various formats. We measure the effectiveness of this feedback in patching these rationales through in-context learning.
• Score: 53.4840989321394
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Eliciting feedback from end users of NLP models can be beneficial for improving models. However, how should we present model responses to users so they are most amenable to be corrected from user feedback? Further, what properties do users value to understand and trust responses? We answer these questions by analyzing the effect of rationales (or explanations) generated by QA models to support their answers. We specifically consider decomposed QA models that first extract an intermediate rationale based on a context and a question and then use solely this rationale to answer the question. A rationale outlines the approach followed by the model to answer the question. Our work considers various formats of these rationales that vary according to well-defined properties of interest. We sample rationales from language models using few-shot prompting for two datasets, and then perform two user studies. First, we present users with incorrect answers and corresponding rationales in various formats and ask them to provide natural language feedback to revise the rationale. We then measure the effectiveness of this feedback in patching these rationales through in-context learning. The second study evaluates how well different rationale formats enable users to understand and trust model answers, when they are correct. We find that rationale formats significantly affect how easy it is (1) for users to give feedback for rationales, and (2) for models to subsequently execute this feedback. In addition, formats with attributions to the context and in-depth reasoning significantly enhance user-reported understanding and trust of model outputs.

Related papers

• Contextualized Evaluations: Taking the Guesswork Out of Language Model Evaluations [85.81295563405433]
  Language model users often issue queries that lack specification, where the context under which a query was issued is not explicit. We present contextualized evaluations, a protocol that synthetically constructs context surrounding an under-specified query and provides it during evaluation. We find that the presence of context can 1) alter conclusions drawn from evaluation, even flipping win rates between model pairs, 2) nudge evaluators to make fewer judgments based on surface-level criteria, like style, and 3) provide new insights about model behavior across diverse contexts.
  arXiv  Detail & Related papers  (2024-11-11T18:58:38Z)
• Disentangling Likes and Dislikes in Personalized Generative Explainable Recommendation [26.214148426964794]
  We introduce new datasets and evaluation methods that focus on the users' sentiments. We construct the datasets by explicitly extracting users' positive and negative opinions from their post-purchase reviews. We propose to evaluate systems based on whether the generated explanations align well with the users' sentiments.
  arXiv  Detail & Related papers  (2024-10-17T06:15:00Z)
• When factorization meets argumentation: towards argumentative explanations [0.0]
  We propose a novel model that combines factorization-based methods with argumentation frameworks (AFs) Our framework seamlessly incorporates side information, such as user contexts, leading to more accurate predictions.
  arXiv  Detail & Related papers  (2024-05-13T19:16:28Z)
• Evaluating the Utility of Model Explanations for Model Development [54.23538543168767]
  We evaluate whether explanations can improve human decision-making in practical scenarios of machine learning model development. To our surprise, we did not find evidence of significant improvement on tasks when users were provided with any of the saliency maps. These findings suggest caution regarding the usefulness and potential for misunderstanding in saliency-based explanations.
  arXiv  Detail & Related papers  (2023-12-10T23:13:23Z)
• Getting MoRE out of Mixture of Language Model Reasoning Experts [71.61176122960464]
  We propose a Mixture-of-Reasoning-Experts (MoRE) framework that ensembles diverse specialized language models. We specialize the backbone language model with prompts optimized for different reasoning categories, including factual, multihop, mathematical, and commonsense reasoning. Our human study confirms that presenting expert predictions and the answer selection process helps annotators more accurately calibrate when to trust the system's output.
  arXiv  Detail & Related papers  (2023-05-24T02:00:51Z)
• Do Users Benefit From Interpretable Vision? A User Study, Baseline, And Dataset [8.863479255829139]
  We conduct a user study to test how a baseline explanation technique performs against concept-based and counterfactual explanations. In a study, we assess if participants can identify the relevant set of attributes compared to the ground-truth. Counterfactual explanations from an invertible neural network performed similarly as the baseline.
  arXiv  Detail & Related papers  (2022-04-25T13:20:06Z)
• Not all users are the same: Providing personalized explanations for sequential decision making problems [25.24098967133101]
  This work proposes an end-to-end adaptive explanation generation system. It begins by learning the different types of users that the agent could interact with. It is then tasked with identifying the type on the fly and adjust its explanations accordingly.
  arXiv  Detail & Related papers  (2021-06-23T07:46:19Z)
• Contrastive Explanations for Model Interpretability [77.92370750072831]
  We propose a methodology to produce contrastive explanations for classification models. Our method is based on projecting model representation to a latent space. Our findings shed light on the ability of label-contrastive explanations to provide a more accurate and finer-grained interpretability of a model's decision.
  arXiv  Detail & Related papers  (2021-03-02T00:36:45Z)
• F1 is Not Enough! Models and Evaluation Towards User-Centered Explainable Question Answering [30.95495958937006]
  We show that current models and evaluation settings have shortcomings regarding the coupling of answer and explanation. We propose a hierarchical model and a new regularization term to strengthen the answer-explanation coupling. Our scores are better aligned with user experience, making them promising candidates for model selection.
  arXiv  Detail & Related papers  (2020-10-13T10:53:20Z)
• Leakage-Adjusted Simulatability: Can Models Generate Non-Trivial Explanations of Their Behavior in Natural Language? [86.60613602337246]
  We introduce a leakage-adjusted simulatability (LAS) metric for evaluating NL explanations. LAS measures how well explanations help an observer predict a model's output, while controlling for how explanations can directly leak the output. We frame explanation generation as a multi-agent game and optimize explanations for simulatability while penalizing label leakage.
  arXiv  Detail & Related papers  (2020-10-08T16:59:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.