Trust in AI: Interpretability is not necessary or sufficient, while black-box interaction is necessary and sufficient

• URL: http://arxiv.org/abs/2202.05302v1
• Date: Thu, 10 Feb 2022 19:59:23 GMT
• Title: Trust in AI: Interpretability is not necessary or sufficient, while black-box interaction is necessary and sufficient
• Authors: Max W. Shen
• Abstract summary: The problem of human trust in artificial intelligence is one of the most fundamental problems in applied machine learning. We draw from statistical learning theory and sociological lenses on human-automation trust to motivate an AI-as-tool framework. We clarify the role of interpretability in trust with a ladder of model access.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The problem of human trust in artificial intelligence is one of the most fundamental problems in applied machine learning. Our processes for evaluating AI trustworthiness have substantial ramifications for ML's impact on science, health, and humanity, yet confusion surrounds foundational concepts. What does it mean to trust an AI, and how do humans assess AI trustworthiness? What are the mechanisms for building trustworthy AI? And what is the role of interpretable ML in trust? Here, we draw from statistical learning theory and sociological lenses on human-automation trust to motivate an AI-as-tool framework, which distinguishes human-AI trust from human-AI-human trust. Evaluating an AI's contractual trustworthiness involves predicting future model behavior using behavior certificates (BCs) that aggregate behavioral evidence from diverse sources including empirical out-of-distribution and out-of-task evaluation and theoretical proofs linking model architecture to behavior. We clarify the role of interpretability in trust with a ladder of model access. Interpretability (level 3) is not necessary or even sufficient for trust, while the ability to run a black-box model at-will (level 2) is necessary and sufficient. While interpretability can offer benefits for trust, it can also incur costs. We clarify ways interpretability can contribute to trust, while questioning the perceived centrality of interpretability to trust in popular discourse. How can we empower people with tools to evaluate trust? Instead of trying to understand how a model works, we argue for understanding how a model behaves. Instead of opening up black boxes, we should create more behavior certificates that are more correct, relevant, and understandable. We discuss how to build trusted and trustworthy AI responsibly.

Related papers

• Fostering Trust and Quantifying Value of AI and ML [0.0]
  Much has been discussed about trusting AI and ML inferences, but little has been done to define what that means. producing ever more trustworthy machine learning inferences is a path to increase the value of products.
  arXiv  Detail & Related papers  (2024-07-08T13:25:28Z)
• Trust in AI: Progress, Challenges, and Future Directions [6.724854390957174]
  The increasing use of artificial intelligence (AI) systems in our daily life explains the significance of trust/distrust in AI from a user perspective. Trust/distrust in AI plays the role of a regulator and could significantly control the level of this diffusion.
  arXiv  Detail & Related papers  (2024-03-12T20:26:49Z)
• Towards Machines that Trust: AI Agents Learn to Trust in the Trust Game [11.788352764861369]
  We present a theoretical analysis of the $textittrust game$, the canonical task for studying trust in behavioral and brain sciences. Specifically, leveraging reinforcement learning to train our AI agents, we investigate learning trust under various parameterizations of this task. Our theoretical analysis, corroborated by the simulations results presented, provides a mathematical basis for the emergence of trust in the trust game.
  arXiv  Detail & Related papers  (2023-12-20T09:32:07Z)
• A Diachronic Perspective on User Trust in AI under Uncertainty [52.44939679369428]
  Modern NLP systems are often uncalibrated, resulting in confidently incorrect predictions that undermine user trust. We study the evolution of user trust in response to trust-eroding events using a betting game.
  arXiv  Detail & Related papers  (2023-10-20T14:41:46Z)
• Distrust in (X)AI -- Measurement Artifact or Distinct Construct? [0.0]
  Trust is a key motivation in developing explainable artificial intelligence (XAI) Distrust seems relatively understudied in XAI. psychometric evidence favors a distinction between trust and distrust.
  arXiv  Detail & Related papers  (2023-03-29T07:14:54Z)
• When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment [96.77970239683475]
  AI systems need to be able to understand, interpret and predict human moral judgments and decisions. A central challenge for AI safety is capturing the flexibility of the human moral mind. We present a novel challenge set consisting of rule-breaking question answering.
  arXiv  Detail & Related papers  (2022-10-04T09:04:27Z)
• Designing for Responsible Trust in AI Systems: A Communication Perspective [56.80107647520364]
  We draw from communication theories and literature on trust in technologies to develop a conceptual model called MATCH. We highlight transparency and interaction as AI systems' affordances that present a wide range of trustworthiness cues to users. We propose a checklist of requirements to help technology creators identify appropriate cues to use.
  arXiv  Detail & Related papers  (2022-04-29T00:14:33Z)
• Cybertrust: From Explainable to Actionable and Interpretable AI (AI2) [58.981120701284816]
  Actionable and Interpretable AI (AI2) will incorporate explicit quantifications and visualizations of user confidence in AI recommendations. It will allow examining and testing of AI system predictions to establish a basis for trust in the systems' decision making.
  arXiv  Detail & Related papers  (2022-01-26T18:53:09Z)
• Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI [55.4046755826066]
  We discuss a model of trust inspired by, but not identical to, sociology's interpersonal trust (i.e., trust between people) We incorporate a formalization of 'contractual trust', such that trust between a user and an AI is trust that some implicit or explicit contract will hold. We discuss how to design trustworthy AI, how to evaluate whether trust has manifested, and whether it is warranted.
  arXiv  Detail & Related papers  (2020-10-15T03:07:23Z)
• Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making [53.62514158534574]
  We study whether features that reveal case-specific model information can calibrate trust and improve the joint performance of the human and AI. We show that confidence score can help calibrate people's trust in an AI model, but trust calibration alone is not sufficient to improve AI-assisted decision making.
  arXiv  Detail & Related papers  (2020-01-07T15:33:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.