What if you said that differently?: How Explanation Formats Affect Human Feedback Efficacy and User Perception

• URL: http://arxiv.org/abs/2311.09558v2
• Date: Tue, 2 Apr 2024 00:59:36 GMT
• Title: What if you said that differently?: How Explanation Formats Affect Human Feedback Efficacy and User Perception
• Authors: Chaitanya Malaviya, Subin Lee, Dan Roth, Mark Yatskar,
• Abstract summary: We analyze the effect of rationales generated by QA models to support their answers. We present users with incorrect answers and corresponding rationales in various formats. We measure the effectiveness of this feedback in patching these rationales through in-context learning.
• Score: 53.4840989321394
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Eliciting feedback from end users of NLP models can be beneficial for improving models. However, how should we present model responses to users so they are most amenable to be corrected from user feedback? Further, what properties do users value to understand and trust responses? We answer these questions by analyzing the effect of rationales (or explanations) generated by QA models to support their answers. We specifically consider decomposed QA models that first extract an intermediate rationale based on a context and a question and then use solely this rationale to answer the question. A rationale outlines the approach followed by the model to answer the question. Our work considers various formats of these rationales that vary according to well-defined properties of interest. We sample rationales from language models using few-shot prompting for two datasets, and then perform two user studies. First, we present users with incorrect answers and corresponding rationales in various formats and ask them to provide natural language feedback to revise the rationale. We then measure the effectiveness of this feedback in patching these rationales through in-context learning. The second study evaluates how well different rationale formats enable users to understand and trust model answers, when they are correct. We find that rationale formats significantly affect how easy it is (1) for users to give feedback for rationales, and (2) for models to subsequently execute this feedback. In addition, formats with attributions to the context and in-depth reasoning significantly enhance user-reported understanding and trust of model outputs.

Related papers

• When factorization meets argumentation: towards argumentative explanations [0.0]
  We propose a novel model that combines factorization-based methods with argumentation frameworks (AFs) Our framework seamlessly incorporates side information, such as user contexts, leading to more accurate predictions.
  arXiv  Detail & Related papers  (2024-05-13T19:16:28Z)
• Evaluating the Utility of Model Explanations for Model Development [54.23538543168767]
  We evaluate whether explanations can improve human decision-making in practical scenarios of machine learning model development. To our surprise, we did not find evidence of significant improvement on tasks when users were provided with any of the saliency maps. These findings suggest caution regarding the usefulness and potential for misunderstanding in saliency-based explanations.
  arXiv  Detail & Related papers  (2023-12-10T23:13:23Z)
• InterroLang: Exploring NLP Models and Datasets through Dialogue-based Explanations [8.833264791078825]
  We adapt the conversational explanation framework TalkToModel to the NLP domain, add new NLP-specific operations such as free-text rationalization. To recognize user queries for explanations, we evaluate fine-tuned and few-shot prompting models. We conduct two user studies on (1) the perceived correctness and helpfulness of the dialogues, and (2) the simulatability.
  arXiv  Detail & Related papers  (2023-10-09T10:27:26Z)
• UltraFeedback: Boosting Language Models with Scaled AI Feedback [99.4633351133207]
  We present textscUltraFeedback, a large-scale, high-quality, and diversified AI feedback dataset. Our work validates the effectiveness of scaled AI feedback data in constructing strong open-source chat language models.
  arXiv  Detail & Related papers  (2023-10-02T17:40:01Z)
• Getting MoRE out of Mixture of Language Model Reasoning Experts [71.61176122960464]
  We propose a Mixture-of-Reasoning-Experts (MoRE) framework that ensembles diverse specialized language models. We specialize the backbone language model with prompts optimized for different reasoning categories, including factual, multihop, mathematical, and commonsense reasoning. Our human study confirms that presenting expert predictions and the answer selection process helps annotators more accurately calibrate when to trust the system's output.
  arXiv  Detail & Related papers  (2023-05-24T02:00:51Z)
• Do Users Benefit From Interpretable Vision? A User Study, Baseline, And Dataset [8.863479255829139]
  We conduct a user study to test how a baseline explanation technique performs against concept-based and counterfactual explanations. In a study, we assess if participants can identify the relevant set of attributes compared to the ground-truth. Counterfactual explanations from an invertible neural network performed similarly as the baseline.
  arXiv  Detail & Related papers  (2022-04-25T13:20:06Z)
• Not all users are the same: Providing personalized explanations for sequential decision making problems [25.24098967133101]
  This work proposes an end-to-end adaptive explanation generation system. It begins by learning the different types of users that the agent could interact with. It is then tasked with identifying the type on the fly and adjust its explanations accordingly.
  arXiv  Detail & Related papers  (2021-06-23T07:46:19Z)
• Contrastive Explanations for Model Interpretability [77.92370750072831]
  We propose a methodology to produce contrastive explanations for classification models. Our method is based on projecting model representation to a latent space. Our findings shed light on the ability of label-contrastive explanations to provide a more accurate and finer-grained interpretability of a model's decision.
  arXiv  Detail & Related papers  (2021-03-02T00:36:45Z)
• F1 is Not Enough! Models and Evaluation Towards User-Centered Explainable Question Answering [30.95495958937006]
  We show that current models and evaluation settings have shortcomings regarding the coupling of answer and explanation. We propose a hierarchical model and a new regularization term to strengthen the answer-explanation coupling. Our scores are better aligned with user experience, making them promising candidates for model selection.
  arXiv  Detail & Related papers  (2020-10-13T10:53:20Z)
• Leakage-Adjusted Simulatability: Can Models Generate Non-Trivial Explanations of Their Behavior in Natural Language? [86.60613602337246]
  We introduce a leakage-adjusted simulatability (LAS) metric for evaluating NL explanations. LAS measures how well explanations help an observer predict a model's output, while controlling for how explanations can directly leak the output. We frame explanation generation as a multi-agent game and optimize explanations for simulatability while penalizing label leakage.
  arXiv  Detail & Related papers  (2020-10-08T16:59:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.