How Metacognitive Architectures Remember Their Own Thoughts: A Systematic Review

• URL: http://arxiv.org/abs/2503.13467v1
• Date: Fri, 28 Feb 2025 08:48:41 GMT
• Title: How Metacognitive Architectures Remember Their Own Thoughts: A Systematic Review
• Authors: Robin Nolte, Mihai Pomarlan, Ayden Janssen, Daniel Beßler, Kamyar Javanmardi, Sascha Jongebloed, Robert Porzel, John Bateman, Michael Beetz, Rainer Malaka,
• Abstract summary: We review how Computational Metacognitive Architectures (CMAs) model, store, remember and process their metacognitive experiences.<n>We consider different aspects - ranging from the underlying psychological theories to the content and structure of collected data, to the algorithms used and evaluation results.
• Score: 16.35521789216079
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Inspired by human cognition, metacognition has gained significant attention for its potential to enhance autonomy, adaptability, and robust learning in artificial agents. Yet research on Computational Metacognitive Architectures (CMAs) remains fragmented: diverse theories, terminologies, and design choices have led to disjointed developments and limited comparability across systems. Existing overviews and surveys often remain at a broad, conceptual level, making it difficult to synthesize deeper insights into the underlying algorithms and representations, and their respective success. We address this gap by performing an explorative systematic review of how CMAs model, store, remember and process their metacognitive experiences, one of Flavell's (1979) three foundational components of metacognition. Following this organizing principle, we identify 35 CMAs that feature episodic introspective data ranging from symbolic event traces to sub-symbolic arousal metrics. We consider different aspects - ranging from the underlying psychological theories to the content and structure of collected data, to the algorithms used and evaluation results - and derive a unifying perspective that allows us to compare in depth how different Computational Metacognitive Architectures (CMAs) leverage metacognitive experiences for tasks such as error diagnosis, self-repair, and goal-driven learning. Our findings highlight both the promise of metacognitive experiences - in boosting adaptability, explainability, and overall system performance - and the persistent lack of shared standards or evaluation benchmarks.

Related papers

• Medical Reasoning in the Era of LLMs: A Systematic Review of Enhancement Techniques and Applications [59.721265428780946]
  Large Language Models (LLMs) in medicine have enabled impressive capabilities, yet a critical gap remains in their ability to perform systematic, transparent, and verifiable reasoning.<n>This paper provides the first systematic review of this emerging field.<n>We propose a taxonomy of reasoning enhancement techniques, categorized into training-time strategies and test-time mechanisms.
  arXiv  Detail & Related papers  (2025-08-01T14:41:31Z)
• Anomaly Detection and Generation with Diffusion Models: A Survey [51.61574868316922]
  Anomaly detection (AD) plays a pivotal role across diverse domains, including cybersecurity, finance, healthcare, and industrial manufacturing.<n>Recent advancements in deep learning, specifically diffusion models (DMs), have sparked significant interest.<n>This survey aims to guide researchers and practitioners in leveraging DMs for innovative AD solutions across diverse applications.
  arXiv  Detail & Related papers  (2025-06-11T03:29:18Z)
• A Framework for Robust Cognitive Evaluation of LLMs [13.822169295436177]
  Emergent cognitive abilities in large language models (LLMs) have been widely observed, but their nature and underlying mechanisms remain poorly understood. We develop CognitivEval, a framework for systematically evaluating the artificial cognitive capabilities of LLMs.
  arXiv  Detail & Related papers  (2025-04-03T17:35:54Z)
• A Comprehensive Review on Hashtag Recommendation: From Traditional to Deep Learning and Beyond [0.37865171120254354]
  Hashtags, as a fundamental categorization mechanism, play a pivotal role in enhancing content visibility and user engagement.<n>The development of accurate and robust hashtag recommendation systems remains a complex and evolving research challenge.<n>This review article conducts a systematic analysis of hashtag recommendation systems, examining recent advancements across several dimensions.
  arXiv  Detail & Related papers  (2025-03-24T13:40:36Z)
• VisFactor: Benchmarking Fundamental Visual Cognition in Multimodal Large Language Models [62.667142971664575]
  We introduce VisFactor, a novel benchmark derived from the Factor-Referenced Cognitive Test (FRCT) VisFactor digitalizes vision-related FRCT subtests to systematically evaluate MLLMs across essential visual cognitive tasks. We present a comprehensive evaluation of state-of-the-art MLLMs, such as GPT-4o, Gemini-Pro, and Qwen-VL.
  arXiv  Detail & Related papers  (2025-02-23T04:21:32Z)
• Enhancing Cognitive Diagnosis by Modeling Learner Cognitive Structure State [3.0103051895985256]
  Theoretically, an individual's cognitive state is essentially equivalent to their cognitive structure state.<n>A learner's cognitive structure is essential for promoting meaningful learning and shaping academic performance.
  arXiv  Detail & Related papers  (2024-12-27T17:41:39Z)
• Navigating the Maze of Explainable AI: A Systematic Approach to Evaluating Methods and Metrics [10.045644410833402]
  We introduce LATEC, a large-scale benchmark that critically evaluates 17 prominent XAI methods using 20 distinct metrics.<n>We showcase the high risk of conflicting metrics leading to unreliable rankings and consequently propose a more robust evaluation scheme.<n>LATEC reinforces its role in future XAI research by publicly releasing all 326k saliency maps and 378k metric scores as a (meta-evaluation) dataset.
  arXiv  Detail & Related papers  (2024-09-25T09:07:46Z)
• Coding for Intelligence from the Perspective of Category [66.14012258680992]
  Coding targets compressing and reconstructing data, and intelligence. Recent trends demonstrate the potential homogeneity of these two fields. We propose a novel problem of Coding for Intelligence from the category theory view.
  arXiv  Detail & Related papers  (2024-07-01T07:05:44Z)
• MR-Ben: A Meta-Reasoning Benchmark for Evaluating System-2 Thinking in LLMs [55.20845457594977]
  Large language models (LLMs) have shown increasing capability in problem-solving and decision-making.<n>We present a process-based benchmark MR-Ben that demands a meta-reasoning skill.<n>Our meta-reasoning paradigm is especially suited for system-2 slow thinking.
  arXiv  Detail & Related papers  (2024-06-20T03:50:23Z)
• GenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation Models [56.63218531256961]
  We introduce GenBench, a benchmarking suite specifically tailored for evaluating the efficacy of Genomic Foundation Models. GenBench offers a modular and expandable framework that encapsulates a variety of state-of-the-art methodologies. We provide a nuanced analysis of the interplay between model architecture and dataset characteristics on task-specific performance.
  arXiv  Detail & Related papers  (2024-06-01T08:01:05Z)
• A Survey of Automatic Hallucination Evaluation on Natural Language Generation [18.277552023139847]
  Large Language Models (LLMs) have introduced a critical challenge: accurate hallucination evaluation.<n>This survey addresses this critical gap through a comprehensive analysis of 74 evaluation methods.<n>We propose strategic directions, including enhanced interpretability mechanisms and integration of application-specific evaluation criteria.
  arXiv  Detail & Related papers  (2024-04-18T09:52:18Z)
• What Does Evaluation of Explainable Artificial Intelligence Actually Tell Us? A Case for Compositional and Contextual Validation of XAI Building Blocks [16.795332276080888]
  We propose a fine-grained validation framework for explainable artificial intelligence systems. We recognise their inherent modular structure: technical building blocks, user-facing explanatory artefacts and social communication protocols.
  arXiv  Detail & Related papers  (2024-03-19T13:45:34Z)
• MR-GSM8K: A Meta-Reasoning Benchmark for Large Language Model Evaluation [60.65820977963331]
  We introduce a novel evaluation paradigm for Large Language Models (LLMs) This paradigm shifts the emphasis from result-oriented assessments, which often neglect the reasoning process, to a more comprehensive evaluation. By applying this paradigm in the GSM8K dataset, we have developed the MR-GSM8K benchmark.
  arXiv  Detail & Related papers  (2023-12-28T15:49:43Z)
• Hierarchical Audio-Visual Information Fusion with Multi-label Joint Decoding for MER 2023 [51.95161901441527]
  In this paper, we propose a novel framework for recognizing both discrete and dimensional emotions. Deep features extracted from foundation models are used as robust acoustic and visual representations of raw video. Our final system achieves state-of-the-art performance and ranks third on the leaderboard on MER-MULTI sub-challenge.
  arXiv  Detail & Related papers  (2023-09-11T03:19:10Z)
• Bias and Fairness in Large Language Models: A Survey [73.87651986156006]
  We present a comprehensive survey of bias evaluation and mitigation techniques for large language models (LLMs) We first consolidate, formalize, and expand notions of social bias and fairness in natural language processing. We then unify the literature by proposing three intuitive, two for bias evaluation, and one for mitigation.
  arXiv  Detail & Related papers  (2023-09-02T00:32:55Z)
• OlaGPT: Empowering LLMs With Human-like Problem-Solving Abilities [19.83434949066066]
  This paper introduces a novel intelligent framework, referred to as OlaGPT. OlaGPT carefully studied a cognitive architecture framework, and propose to simulate certain aspects of human cognition. The framework involves approximating different cognitive modules, including attention, memory, reasoning, learning, and corresponding scheduling and decision-making mechanisms.
  arXiv  Detail & Related papers  (2023-05-23T09:36:51Z)
• Computational Metacognition [2.0552049801885746]
  Computational metacognition represents a cognitive systems perspective on high-order reasoning in integrated artificial systems. We show how computational metacognition improves performance by changing cognition through meta-level goal operations and learning.
  arXiv  Detail & Related papers  (2022-01-30T17:34:53Z)
• Confounder Identification-free Causal Visual Feature Learning [84.28462256571822]
  We propose a novel Confounder Identification-free Causal Visual Feature Learning (CICF) method, which obviates the need for identifying confounders. CICF models the interventions among different samples based on front-door criterion, and then approximates the global-scope intervening effect upon the instance-level interventions. We uncover the relation between CICF and the popular meta-learning strategy MAML, and provide an interpretation of why MAML works from the theoretical perspective.
  arXiv  Detail & Related papers  (2021-11-26T10:57:47Z)
• Towards Unbiased Visual Emotion Recognition via Causal Intervention [63.74095927462]
  We propose a novel Emotion Recognition Network (IERN) to alleviate the negative effects brought by the dataset bias. A series of designed tests validate the effectiveness of IERN, and experiments on three emotion benchmarks demonstrate that IERN outperforms other state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-07-26T10:40:59Z)
• CogME: A Cognition-Inspired Multi-Dimensional Evaluation Metric for Story Understanding [19.113385429326808]
  We introduce CogME, a cognition-inspired, multi-dimensional evaluation metric designed for AI models focusing on story understanding. We argue the need for metrics based on understanding the nature of tasks and designed to align closely with human cognitive processes. This approach provides insights beyond traditional overall scores and paves the way for more sophisticated AI development targeting higher cognitive functions.
  arXiv  Detail & Related papers  (2021-07-21T02:33:37Z)
• A Minimalist Dataset for Systematic Generalization of Perception, Syntax, and Semantics [131.93113552146195]
  We present a new dataset, Handwritten arithmetic with INTegers (HINT), to examine machines' capability of learning generalizable concepts. In HINT, machines are tasked with learning how concepts are perceived from raw signals such as images. We undertake extensive experiments with various sequence-to-sequence models, including RNNs, Transformers, and GPT-3.
  arXiv  Detail & Related papers  (2021-03-02T01:32:54Z)
• Concept Learners for Few-Shot Learning [76.08585517480807]
  We propose COMET, a meta-learning method that improves generalization ability by learning to learn along human-interpretable concept dimensions. We evaluate our model on few-shot tasks from diverse domains, including fine-grained image classification, document categorization and cell type annotation.
  arXiv  Detail & Related papers  (2020-07-14T22:04:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.