On the consistent reasoning paradox of intelligence and optimal trust in AI: The power of 'I don't know'

• URL: http://arxiv.org/abs/2408.02357v1
• Date: Mon, 5 Aug 2024 10:06:53 GMT
• Title: On the consistent reasoning paradox of intelligence and optimal trust in AI: The power of 'I don't know'
• Authors: Alexander Bastounis, Paolo Campodonico, Mihaela van der Schaar, Ben Adcock, Anders C. Hansen,
• Abstract summary: Consistent reasoning, which lies at the core of human intelligence, is the ability to handle tasks that are equivalent. CRP asserts that consistent reasoning implies fallibility -- in particular, human-like intelligence in AI necessarily comes with human-like fallibility.
• Score: 79.69412622010249
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce the Consistent Reasoning Paradox (CRP). Consistent reasoning, which lies at the core of human intelligence, is the ability to handle tasks that are equivalent, yet described by different sentences ('Tell me the time!' and 'What is the time?'). The CRP asserts that consistent reasoning implies fallibility -- in particular, human-like intelligence in AI necessarily comes with human-like fallibility. Specifically, it states that there are problems, e.g. in basic arithmetic, where any AI that always answers and strives to mimic human intelligence by reasoning consistently will hallucinate (produce wrong, yet plausible answers) infinitely often. The paradox is that there exists a non-consistently reasoning AI (which therefore cannot be on the level of human intelligence) that will be correct on the same set of problems. The CRP also shows that detecting these hallucinations, even in a probabilistic sense, is strictly harder than solving the original problems, and that there are problems that an AI may answer correctly, but it cannot provide a correct logical explanation for how it arrived at the answer. Therefore, the CRP implies that any trustworthy AI (i.e., an AI that never answers incorrectly) that also reasons consistently must be able to say 'I don't know'. Moreover, this can only be done by implicitly computing a new concept that we introduce, termed the 'I don't know' function -- something currently lacking in modern AI. In view of these insights, the CRP also provides a glimpse into the behaviour of Artificial General Intelligence (AGI). An AGI cannot be 'almost sure', nor can it always explain itself, and therefore to be trustworthy it must be able to say 'I don't know'.

Related papers

• Imagining and building wise machines: The centrality of AI metacognition [78.76893632793497]
  We argue that shortcomings stem from one overarching failure: AI systems lack wisdom. While AI research has focused on task-level strategies, metacognition is underdeveloped in AI systems. We propose that integrating metacognitive capabilities into AI systems is crucial for enhancing their robustness, explainability, cooperation, and safety.
  arXiv  Detail & Related papers  (2024-11-04T18:10:10Z)
• Unexplainability of Artificial Intelligence Judgments in Kant's Perspective [0.0]
  This paper argues that AI judgments exhibit a form that cannot be understood in terms of the characteristics of human judgments according to Kant. I show that concepts without physical predicates are not easy to explain when their functions are shown through vision.
  arXiv  Detail & Related papers  (2024-07-12T03:39:55Z)
• Making AI Intelligible: Philosophical Foundations [0.0]
  'Making AI Intelligible' shows that philosophical work on the metaphysics of meaning can help answer these questions. Author: The questions addressed in the book are not only theoretically interesting, but the answers have pressing practical implications.
  arXiv  Detail & Related papers  (2024-06-12T12:25:04Z)
• The Generative AI Paradox: "What It Can Create, It May Not Understand" [81.89252713236746]
  Recent wave of generative AI has sparked excitement and concern over potentially superhuman levels of artificial intelligence. At the same time, models still show basic errors in understanding that would not be expected even in non-expert humans. This presents us with an apparent paradox: how do we reconcile seemingly superhuman capabilities with the persistence of errors that few humans would make?
  arXiv  Detail & Related papers  (2023-10-31T18:07:07Z)
• Getting from Generative AI to Trustworthy AI: What LLMs might learn from Cyc [0.0]
  Generative AI, the most popular current approach to AI, consists of large language models (LLMs) that are trained to produce outputs that are plausible, but not necessarily correct. We discuss an alternative approach to AI which could theoretically address many of the limitations associated with current approaches.
  arXiv  Detail & Related papers  (2023-07-31T16:29:28Z)
• Epistemic considerations when AI answers questions for us [0.0]
  We argue that careless reliance on AI to answer our questions and to judge our output is a violation of Grice's Maxim of Quality and Lemoine's legal Maxim of Innocence. What is missing in the focus on output and results of AI-generated and AI-evaluated content is, apart from paying proper tribute, the demand to follow a person's thought process.
  arXiv  Detail & Related papers  (2023-04-23T08:26:42Z)
• Cybertrust: From Explainable to Actionable and Interpretable AI (AI2) [58.981120701284816]
  Actionable and Interpretable AI (AI2) will incorporate explicit quantifications and visualizations of user confidence in AI recommendations. It will allow examining and testing of AI system predictions to establish a basis for trust in the systems' decision making.
  arXiv  Detail & Related papers  (2022-01-26T18:53:09Z)
• An argument for the impossibility of machine intelligence [0.0]
  We define what it is to be an agent (device) that could be the bearer of AI. We show that the mainstream definitions of intelligence' are too weak even to capture what is involved when we ascribe intelligence to an insect. We identify the properties that an AI agent would need to possess in order to be the bearer of intelligence by this definition.
  arXiv  Detail & Related papers  (2021-10-20T08:54:48Z)
• Trustworthy AI: A Computational Perspective [54.80482955088197]
  We focus on six of the most crucial dimensions in achieving trustworthy AI: (i) Safety & Robustness, (ii) Non-discrimination & Fairness, (iii) Explainability, (iv) Privacy, (v) Accountability & Auditability, and (vi) Environmental Well-Being. For each dimension, we review the recent related technologies according to a taxonomy and summarize their applications in real-world systems.
  arXiv  Detail & Related papers  (2021-07-12T14:21:46Z)
• Inductive Biases for Deep Learning of Higher-Level Cognition [108.89281493851358]
  A fascinating hypothesis is that human and animal intelligence could be explained by a few principles. This work considers a larger list, focusing on those which concern mostly higher-level and sequential conscious processing. The objective of clarifying these particular principles is that they could potentially help us build AI systems benefiting from humans' abilities.
  arXiv  Detail & Related papers  (2020-11-30T18:29:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.