Are we measuring trust correctly in explainability, interpretability, and transparency research?

• URL: http://arxiv.org/abs/2209.00651v1
• Date: Wed, 31 Aug 2022 07:41:08 GMT
• Title: Are we measuring trust correctly in explainability, interpretability, and transparency research?
• Authors: Tim Miller
• Abstract summary: This paper showcases three methods that do a good job at measuring perceived and demonstrated trust. It is intended to be starting point for discussion on this topic, rather than to be the final say.
• Score: 4.452019519213712
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper presents an argument for why we are not measuring trust sufficiently in explainability, interpretability, and transparency research. Most studies ask participants to complete a trust scale to rate their trust of a model that has been explained/interpreted. If the trust is increased, we consider this a positive. However, there are two issues with this. First, we usually have no way of knowing whether participants should trust the model. Trust should surely decrease if a model is of poor quality. Second, these scales measure perceived trust rather than demonstrated trust. This paper showcases three methods that do a good job at measuring perceived and demonstrated trust. It is intended to be starting point for discussion on this topic, rather than to be the final say. The author invites critique and discussion.

Related papers

• Language Models Prefer What They Know: Relative Confidence Estimation via Confidence Preferences [62.52739672949452]
  Language models (LMs) should provide reliable confidence estimates to help users detect mistakes in their outputs and defer to human experts when necessary. We propose relative confidence estimation, where we match up questions against each other and ask the model to make relative judgments of confidence. Treating each question as a "player" in a series of matchups against other questions and the model's preferences as match outcomes, we can use rank aggregation methods like Elo rating and Bradley-Terry to translate the model's confidence preferences into confidence scores.
  arXiv  Detail & Related papers  (2025-02-03T07:43:27Z)
• Correctness is not Faithfulness in RAG Attributions [47.48625339105129]
  Explicitly citing source documents allows users to verify generated responses and increases trust. Prior work largely evaluates citation correctness - whether cited documents support the corresponding statements. To establish trust in attributed answers, we must examine both citation correctness and citation faithfulness.
  arXiv  Detail & Related papers  (2024-12-23T21:57:11Z)
• LACIE: Listener-Aware Finetuning for Confidence Calibration in Large Language Models [69.68379406317682]
  We introduce a listener-aware finetuning method (LACIE) to calibrate implicit and explicit confidence markers. We show that LACIE models the listener, considering not only whether an answer is right, but whether it will be accepted by a listener. We find that training with LACIE results in 47% fewer incorrect answers being accepted while maintaining the same level of acceptance for correct answers.
  arXiv  Detail & Related papers  (2024-05-31T17:16:38Z)
• Distrust in (X)AI -- Measurement Artifact or Distinct Construct? [0.0]
  Trust is a key motivation in developing explainable artificial intelligence (XAI) Distrust seems relatively understudied in XAI. psychometric evidence favors a distinction between trust and distrust.
  arXiv  Detail & Related papers  (2023-03-29T07:14:54Z)
• Contextual Trust [0.0]
  I examine the nature of trust from a philosophical perspective. I propose to view trust as a context-sensitive state in a manner that will be made precise.
  arXiv  Detail & Related papers  (2023-03-15T19:34:58Z)
• Trust and Reliance in XAI -- Distinguishing Between Attitudinal and Behavioral Measures [0.0]
  Researchers argue that AI should be more transparent to increase trust, making transparency one of the main goals of XAI. empirical research on this topic is inconclusive regarding the effect of transparency on trust. We advocate for a clear distinction between behavioral (objective) measures of reliance and attitudinal (subjective) measures of trust.
  arXiv  Detail & Related papers  (2022-03-23T10:39:39Z)
• On the Relation of Trust and Explainability: Why to Engineer for Trustworthiness [0.0]
  One of the primary motivators for such requirements is that explainability is expected to facilitate stakeholders' trust in a system. Recent psychological studies indicate that explanations do not necessarily facilitate trust. We argue that even though trustworthiness does not automatically lead to trust, there are several reasons to engineer primarily for trustworthiness.
  arXiv  Detail & Related papers  (2021-08-11T18:02:08Z)
• Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI [55.4046755826066]
  We discuss a model of trust inspired by, but not identical to, sociology's interpersonal trust (i.e., trust between people) We incorporate a formalization of 'contractual trust', such that trust between a user and an AI is trust that some implicit or explicit contract will hold. We discuss how to design trustworthy AI, how to evaluate whether trust has manifested, and whether it is warranted.
  arXiv  Detail & Related papers  (2020-10-15T03:07:23Z)
• Where Does Trust Break Down? A Quantitative Trust Analysis of Deep Neural Networks via Trust Matrix and Conditional Trust Densities [94.65749466106664]
  We introduce the concept of trust matrix, a novel trust quantification strategy. A trust matrix defines the expected question-answer trust for a given actor-oracle answer scenario. We further extend the concept of trust densities with the notion of conditional trust densities.
  arXiv  Detail & Related papers  (2020-09-30T14:33:43Z)
• How Much Can We Really Trust You? Towards Simple, Interpretable Trust Quantification Metrics for Deep Neural Networks [94.65749466106664]
  We conduct a thought experiment and explore two key questions about trust in relation to confidence. We introduce a suite of metrics for assessing the overall trustworthiness of deep neural networks based on their behaviour when answering a set of questions. The proposed metrics are by no means perfect, but the hope is to push the conversation towards better metrics.
  arXiv  Detail & Related papers  (2020-09-12T17:37:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.