A Unifying Bayesian Formulation of Measures of Interpretability in Human-AI

• URL: http://arxiv.org/abs/2104.10743v1
• Date: Wed, 21 Apr 2021 20:06:33 GMT
• Title: A Unifying Bayesian Formulation of Measures of Interpretability in Human-AI
• Authors: Sarath Sreedharan, Anagha Kulkarni, David E. Smith, Subbarao Kambhampati
• Abstract summary: We present a unifying Bayesian framework that models a human observer's evolving beliefs about an agent. We show that the definitions of interpretability measures like explicability, legibility and predictability from the prior literature fall out as special cases of our general framework.
• Score: 25.239891076153025
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing approaches for generating human-aware agent behaviors have considered different measures of interpretability in isolation. Further, these measures have been studied under differing assumptions, thus precluding the possibility of designing a single framework that captures these measures under the same assumptions. In this paper, we present a unifying Bayesian framework that models a human observer's evolving beliefs about an agent and thereby define the problem of Generalized Human-Aware Planning. We will show that the definitions of interpretability measures like explicability, legibility and predictability from the prior literature fall out as special cases of our general framework. Through this framework, we also bring a previously ignored fact to light that the human-robot interactions are in effect open-world problems, particularly as a result of modeling the human's beliefs over the agent. Since the human may not only hold beliefs unknown to the agent but may also form new hypotheses about the agent when presented with novel or unexpected behaviors.

Related papers

• Performative Prediction on Games and Mechanism Design [69.7933059664256]
  We study a collective risk dilemma where agents decide whether to trust predictions based on past accuracy. As predictions shape collective outcomes, social welfare arises naturally as a metric of concern. We show how to achieve better trade-offs and use them for mechanism design.
  arXiv  Detail & Related papers  (2024-08-09T16:03:44Z)
• Approximating Human Models During Argumentation-based Dialogues [4.178382980763478]
  Key challenge in Explainable AI Planning (XAIP) is model reconciliation. We propose a novel framework that enables AI agents to learn and update a probabilistic human model.
  arXiv  Detail & Related papers  (2024-05-28T23:22:18Z)
• The Essential Role of Causality in Foundation World Models for Embodied AI [102.75402420915965]
  Embodied AI agents will require the ability to perform new tasks in many different real-world environments. Current foundation models fail to accurately model physical interactions and are therefore insufficient for Embodied AI. The study of causality lends itself to the construction of veridical world models.
  arXiv  Detail & Related papers  (2024-02-06T17:15:33Z)
• Ground(less) Truth: A Causal Framework for Proxy Labels in Human-Algorithm Decision-Making [29.071173441651734]
  We identify five sources of target variable bias that can impact the validity of proxy labels in human-AI decision-making tasks. We develop a causal framework to disentangle the relationship between each bias. We conclude by discussing opportunities to better address target variable bias in future research.
  arXiv  Detail & Related papers  (2023-02-13T16:29:11Z)
• What Should I Know? Using Meta-gradient Descent for Predictive Feature Discovery in a Single Stream of Experience [63.75363908696257]
  computational reinforcement learning seeks to construct an agent's perception of the world through predictions of future sensations. An open challenge in this line of work is determining from the infinitely many predictions that the agent could possibly make which predictions might best support decision-making. We introduce a meta-gradient descent process by which an agent learns what predictions to make, 2) the estimates for its chosen predictions, and 3) how to use those estimates to generate policies that maximize future reward.
  arXiv  Detail & Related papers  (2022-06-13T21:31:06Z)
• Empirical Estimates on Hand Manipulation are Recoverable: A Step Towards Individualized and Explainable Robotic Support in Everyday Activities [80.37857025201036]
  Key challenge for robotic systems is to figure out the behavior of another agent. Processing correct inferences is especially challenging when (confounding) factors are not controlled experimentally. We propose equipping robots with the necessary tools to conduct observational studies on people.
  arXiv  Detail & Related papers  (2022-01-27T22:15:56Z)
• Heterogeneous-Agent Trajectory Forecasting Incorporating Class Uncertainty [54.88405167739227]
  We present HAICU, a method for heterogeneous-agent trajectory forecasting that explicitly incorporates agents' class probabilities. We additionally present PUP, a new challenging real-world autonomous driving dataset. We demonstrate that incorporating class probabilities in trajectory forecasting significantly improves performance in the face of uncertainty.
  arXiv  Detail & Related papers  (2021-04-26T10:28:34Z)
• A Bayesian Account of Measures of Interpretability in Human-AI Interaction [34.99424576619341]
  Existing approaches for the design of interpretable agent behavior consider different measures of interpretability in isolation. We propose a revised model where all these behaviors can be meaningfully modeled together. We will highlight interesting consequences of this unified model and motivate, through results of a user study.
  arXiv  Detail & Related papers  (2020-11-22T03:28:28Z)
• A General Framework for Distributed Inference with Uncertain Models [14.8884251609335]
  We study the problem of distributed classification with a network of heterogeneous agents. We build upon the concept of uncertain models to incorporate the agents' uncertainty in the likelihoods.
  arXiv  Detail & Related papers  (2020-11-20T22:17:12Z)
• When Does Uncertainty Matter?: Understanding the Impact of Predictive Uncertainty in ML Assisted Decision Making [68.19284302320146]
  We carry out user studies to assess how people with differing levels of expertise respond to different types of predictive uncertainty. We found that showing posterior predictive distributions led to smaller disagreements with the ML model's predictions. This suggests that posterior predictive distributions can potentially serve as useful decision aids which should be used with caution and take into account the type of distribution and the expertise of the human.
  arXiv  Detail & Related papers  (2020-11-12T02:23:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.