Human Uncertainty in Concept-Based AI Systems
- URL: http://arxiv.org/abs/2303.12872v1
- Date: Wed, 22 Mar 2023 19:17:57 GMT
- Title: Human Uncertainty in Concept-Based AI Systems
- Authors: Katherine M. Collins, Matthew Barker, Mateo Espinosa Zarlenga, Naveen
Raman, Umang Bhatt, Mateja Jamnik, Ilia Sucholutsky, Adrian Weller,
Krishnamurthy Dvijotham
- Abstract summary: We study human uncertainty in the context of concept-based AI systems.
We show that training with uncertain concept labels may help mitigate weaknesses in concept-based systems.
- Score: 37.82747673914624
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Placing a human in the loop may abate the risks of deploying AI systems in
safety-critical settings (e.g., a clinician working with a medical AI system).
However, mitigating risks arising from human error and uncertainty within such
human-AI interactions is an important and understudied issue. In this work, we
study human uncertainty in the context of concept-based models, a family of AI
systems that enable human feedback via concept interventions where an expert
intervenes on human-interpretable concepts relevant to the task. Prior work in
this space often assumes that humans are oracles who are always certain and
correct. Yet, real-world decision-making by humans is prone to occasional
mistakes and uncertainty. We study how existing concept-based models deal with
uncertain interventions from humans using two novel datasets: UMNIST, a visual
dataset with controlled simulated uncertainty based on the MNIST dataset, and
CUB-S, a relabeling of the popular CUB concept dataset with rich,
densely-annotated soft labels from humans. We show that training with uncertain
concept labels may help mitigate weaknesses of concept-based systems when
handling uncertain interventions. These results allow us to identify several
open challenges, which we argue can be tackled through future multidisciplinary
research on building interactive uncertainty-aware systems. To facilitate
further research, we release a new elicitation platform, UElic, to collect
uncertain feedback from humans in collaborative prediction tasks.
Related papers
- Imagining and building wise machines: The centrality of AI metacognition [78.76893632793497]
We argue that shortcomings stem from one overarching failure: AI systems lack wisdom.
While AI research has focused on task-level strategies, metacognition is underdeveloped in AI systems.
We propose that integrating metacognitive capabilities into AI systems is crucial for enhancing their robustness, explainability, cooperation, and safety.
arXiv Detail & Related papers (2024-11-04T18:10:10Z) - Human-Modeling in Sequential Decision-Making: An Analysis through the Lens of Human-Aware AI [20.21053807133341]
We try to provide an account of what constitutes a human-aware AI system.
We see that human-aware AI is a design oriented paradigm, one that focuses on the need for modeling the humans it may interact with.
arXiv Detail & Related papers (2024-05-13T14:17:52Z) - Closely Interactive Human Reconstruction with Proxemics and Physics-Guided Adaption [64.07607726562841]
Existing multi-person human reconstruction approaches mainly focus on recovering accurate poses or avoiding penetration.
In this work, we tackle the task of reconstructing closely interactive humans from a monocular video.
We propose to leverage knowledge from proxemic behavior and physics to compensate the lack of visual information.
arXiv Detail & Related papers (2024-04-17T11:55:45Z) - Enabling High-Level Machine Reasoning with Cognitive Neuro-Symbolic
Systems [67.01132165581667]
We propose to enable high-level reasoning in AI systems by integrating cognitive architectures with external neuro-symbolic components.
We illustrate a hybrid framework centered on ACT-R and we discuss the role of generative models in recent and future applications.
arXiv Detail & Related papers (2023-11-13T21:20:17Z) - Intent-aligned AI systems deplete human agency: the need for agency
foundations research in AI safety [2.3572498744567127]
We argue that alignment to human intent is insufficient for safe AI systems.
We argue that preservation of long-term agency of humans may be a more robust standard.
arXiv Detail & Related papers (2023-05-30T17:14:01Z) - Deep Interpretable Models of Theory of Mind For Human-Agent Teaming [0.7734726150561086]
We develop an interpretable modular neural framework for modeling the intentions of other observed entities.
We demonstrate the efficacy of our approach with experiments on data from human participants on a search and rescue task in Minecraft.
arXiv Detail & Related papers (2021-04-07T06:18:58Z) - Adversarial Interaction Attack: Fooling AI to Misinterpret Human
Intentions [46.87576410532481]
We show that, despite their current huge success, deep learning based AI systems can be easily fooled by subtle adversarial noise.
Based on a case study of skeleton-based human interactions, we propose a novel adversarial attack on interactions.
Our study highlights potential risks in the interaction loop with AI and humans, which need to be carefully addressed when deploying AI systems in safety-critical applications.
arXiv Detail & Related papers (2021-01-17T16:23:20Z) - Modelos din\^amicos aplicados \`a aprendizagem de valores em
intelig\^encia artificial [0.0]
Several researchers in the area have developed a robust, beneficial, and safe concept of AI for the preservation of humanity and the environment.
It is utmost importance that artificial intelligent agents have their values aligned with human values.
Perhaps this difficulty comes from the way we are addressing the problem of expressing values using cognitive methods.
arXiv Detail & Related papers (2020-07-30T00:56:11Z) - An Uncertainty-based Human-in-the-loop System for Industrial Tool Wear
Analysis [68.8204255655161]
We show that uncertainty measures based on Monte-Carlo dropout in the context of a human-in-the-loop system increase the system's transparency and performance.
A simulation study demonstrates that the uncertainty-based human-in-the-loop system increases performance for different levels of human involvement.
arXiv Detail & Related papers (2020-07-14T15:47:37Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.