An Algorithmic Theory of Metacognition in Minds and Machines
- URL: http://arxiv.org/abs/2111.03745v1
- Date: Fri, 5 Nov 2021 22:31:09 GMT
- Title: An Algorithmic Theory of Metacognition in Minds and Machines
- Authors: Rylan Schaeffer
- Abstract summary: We present an algorithmic theory of metacognition based on a well-understood trade-off in reinforcement learning.
We show how to create metacognition in machines by implementing a deep MAC.
- Score: 1.52292571922932
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Humans sometimes choose actions that they themselves can identify as
sub-optimal, or wrong, even in the absence of additional information. How is
this possible? We present an algorithmic theory of metacognition based on a
well-understood trade-off in reinforcement learning (RL) between value-based RL
and policy-based RL. To the cognitive (neuro)science community, our theory
answers the outstanding question of why information can be used for error
detection but not for action selection. To the machine learning community, our
proposed theory creates a novel interaction between the Actor and Critic in
Actor-Critic agents and notes a novel connection between RL and Bayesian
Optimization. We call our proposed agent the Metacognitive Actor Critic (MAC).
We conclude with showing how to create metacognition in machines by
implementing a deep MAC and showing that it can detect (some of) its own
suboptimal actions without external information or delay.
Related papers
- Semifactual Explanations for Reinforcement Learning [1.5320737596132754]
Reinforcement Learning (RL) is a learning paradigm in which the agent learns from its environment through trial and error.
Deep reinforcement learning (DRL) algorithms represent the agent's policies using neural networks, making their decisions difficult to interpret.
Explaining the behaviour of DRL agents is necessary to advance user trust, increase engagement, and facilitate integration with real-life tasks.
arXiv Detail & Related papers (2024-09-09T08:37:47Z) - Interactive Autonomous Navigation with Internal State Inference and
Interactivity Estimation [58.21683603243387]
We propose three auxiliary tasks with relational-temporal reasoning and integrate them into the standard Deep Learning framework.
These auxiliary tasks provide additional supervision signals to infer the behavior patterns other interactive agents.
Our approach achieves robust and state-of-the-art performance in terms of standard evaluation metrics.
arXiv Detail & Related papers (2023-11-27T18:57:42Z) - GANterfactual-RL: Understanding Reinforcement Learning Agents'
Strategies through Visual Counterfactual Explanations [0.7874708385247353]
We propose a novel but simple method to generate counterfactual explanations for RL agents.
Our method is fully model-agnostic and we demonstrate that it outperforms the only previous method in several computational metrics.
arXiv Detail & Related papers (2023-02-24T15:29:43Z) - Pessimism meets VCG: Learning Dynamic Mechanism Design via Offline
Reinforcement Learning [114.36124979578896]
We design a dynamic mechanism using offline reinforcement learning algorithms.
Our algorithm is based on the pessimism principle and only requires a mild assumption on the coverage of the offline data set.
arXiv Detail & Related papers (2022-05-05T05:44:26Z) - Automated Machine Learning, Bounded Rationality, and Rational
Metareasoning [62.997667081978825]
We will look at automated machine learning (AutoML) and related problems from the perspective of bounded rationality.
Taking actions under bounded resources requires an agent to reflect on how to use these resources in an optimal way.
arXiv Detail & Related papers (2021-09-10T09:10:20Z) - Interpretable Reinforcement Learning Inspired by Piaget's Theory of
Cognitive Development [1.7778609937758327]
This paper entertains the idea that theories such as language of thought hypothesis (LOTH), script theory, and Piaget's cognitive development theory provide complementary approaches.
The proposed framework can be viewed as a step towards achieving human-like cognition in artificial intelligent systems.
arXiv Detail & Related papers (2021-02-01T00:29:01Z) - Explainability in Deep Reinforcement Learning [68.8204255655161]
We review recent works in the direction to attain Explainable Reinforcement Learning (XRL)
In critical situations where it is essential to justify and explain the agent's behaviour, better explainability and interpretability of RL models could help gain scientific insight on the inner workings of what is still considered a black box.
arXiv Detail & Related papers (2020-08-15T10:11:42Z) - Discovering Reinforcement Learning Algorithms [53.72358280495428]
Reinforcement learning algorithms update an agent's parameters according to one of several possible rules.
This paper introduces a new meta-learning approach that discovers an entire update rule.
It includes both 'what to predict' (e.g. value functions) and 'how to learn from it' by interacting with a set of environments.
arXiv Detail & Related papers (2020-07-17T07:38:39Z) - Emergence of Pragmatics from Referential Game between Theory of Mind
Agents [64.25696237463397]
We propose an algorithm, using which agents can spontaneously learn the ability to "read between lines" without any explicit hand-designed rules.
We integrate the theory of mind (ToM) in a cooperative multi-agent pedagogical situation and propose an adaptive reinforcement learning (RL) algorithm to develop a communication protocol.
arXiv Detail & Related papers (2020-01-21T19:37:33Z) - Making Sense of Reinforcement Learning and Probabilistic Inference [15.987913388420667]
Reinforcement learning (RL) combines a control problem with statistical estimation.
We show that the popular RL as inference' approximation can perform poorly in even very basic problems.
We show that with a small modification the framework does yield algorithms that can provably perform well.
arXiv Detail & Related papers (2020-01-03T12:50:42Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.