Multi-Dimensional Assessment for AI Cognition (MAAC): A Theoretical Framework for Process-Oriented Cognitive Evaluation of Text-Based AI Systems

• URL: http://arxiv.org/abs/2608.00680v1
• Date: Sat, 01 Aug 2026 14:01:17 GMT
• Title: Multi-Dimensional Assessment for AI Cognition (MAAC): A Theoretical Framework for Process-Oriented Cognitive Evaluation of Text-Based AI Systems
• Abstract summary: This paper introduces the Multi-Dimensional Assessment for AI Cognition (MAAC), a framework for shifting evaluation from what text-based AI systems produce to how they think.<n>MAAC defines nine cognitively motivated dimensions: Cognitive Load, Tool Execution, Content Quality, Memory Integration, Complexity Handling, Hallucination Control, Knowledge Transfer, Processing Efficiency, and Process-Outcome Alignment.<n>Five theoretical analyses provide initial support for the framework's coherence and empirical testability.
• Score: 1.8262547855491456
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Evaluating artificial intelligence systems has historically relied on outcome-based benchmarks that measure task accuracy, robustness, or fairness. While indispensable, these benchmarks provide limited diagnostic insight into the underlying cognitive processes that generate performance-leaving critical questions unanswered about how AI systems reason, integrate memory, manage complexity, or avoid generating false information. This paper introduces the Multi-Dimensional Assessment for AI Cognition (MAAC), a theoretically grounded framework for shifting evaluation from what text-based AI systems produce to how they think. MAAC defines nine cognitively motivated dimensions: Cognitive Load, Tool Execution, Content Quality, Memory Integration, Complexity Handling, Hallucination Control, Knowledge Transfer, Processing Efficiency, and Process-Outcome Alignment. Each dimension is grounded in established cognitive science theory-drawing on Marr's tri-level hypothesis, Baddeley's working memory model, Sweller's cognitive load theory, and unified theories of cognition. Five theoretical analyses provide initial support for the framework's coherence and empirical testability: dimension-to-theory mapping; a coverage matrix assessing breadth and non-redundancy; a formal gap analysis relative to current evaluation practice; a worked diagnostic illustration; and a set of a priori interdependency predictions for future empirical testing. MAAC provides a theoretical and operational framework for principled process-level cognitive assessment of text-based AI systems, complementing existing outcome-based benchmarks with cognitively grounded, multi-dimensional evaluation.

Related papers

• Beyond ID Embeddings: Process-Grounded Language Modeling for Cognitive Diagnosis [77.02066500576528]
  Process-aware Language Cognitive Diagnosis (PLCD) framework uses language-derived structures as cognitive priors and response records to calibrate student posterior states.<n>PLCD not only outperforms traditional baselines in predicting student performance but also exhibits strong cognitive transfer capabilities.
  arXiv  Detail & Related papers  (2026-09-11T03:51:20Z)
• A Survey of Reasoning in Autonomous Driving Systems: Open Challenges and Emerging Paradigms [49.66022971508878]
  We argue that reasoning should be elevated from a modular component to the system's cognitive core.<n>We derive and systematize seven core reasoning challenges, such as the responsiveness-reasoning trade-off and social-game reasoning.<n>We identify a fundamental and unresolved tension between the high-latency, deliberative nature of LLM-based reasoning and the millisecond-scale, safety-critical demands of vehicle control.
  arXiv  Detail & Related papers  (2026-03-11T07:40:53Z)
• Measuring What AI Systems Might Do: Towards A Measurement Science in AI [19.687397197326817]
  capabilities, propensities, skills, values, and abilities are routinely used interchangeably and conflated with observable performance.<n>We argue that capabilities and propensities are dispositional properties - stable features of systems characterised by counterfactual relationships.
  arXiv  Detail & Related papers  (2026-02-10T12:59:41Z)
• Theory Trace Card: Theory-Driven Socio-Cognitive Evaluation of LLMs [2.98033672654447]
  We argue that many socio-cognitive evaluations proceed without an explicit theoretical specification of the target capability.<n>Without this theoretical grounding, benchmarks that exercise only narrow subsets of a capability are routinely misinterpreted as evidence of broad competence.<n>We introduce the Trace Card, a lightweight documentation artifact designed to accompany socio-cognitive evaluations.
  arXiv  Detail & Related papers  (2026-01-05T08:06:50Z)
• The Vibe-Check Protocol: Quantifying Cognitive Offloading in AI Programming [5.584060970507507]
  Vibe Coding'' is a paradigm where developers articulate high-level intent through natural language and delegate implementation to AI agents.<n>This paper proposes a theoretical framework to investigate the research question: textitIs Vibe Coding a better way to learn software engineering?
  arXiv  Detail & Related papers  (2026-01-02T06:13:41Z)
• The Artificial Intelligence Cognitive Examination: A Survey on the Evolution of Multimodal Evaluation from Recognition to Reasoning [0.0]
  We argue that the field is undergoing a paradigm shift, moving from simple recognition tasks to complex reasoning benchmarks.<n>We chart the journey from the foundational "knowledge tests" of the ImageNet era to the "applied logic and comprehension" exams.<n>We explore the uncharted territories of evaluating abstract, creative, and social intelligence.
  arXiv  Detail & Related papers  (2025-10-05T10:41:22Z)
• The next question after Turing's question: Introducing the Grow-AI test [51.56484100374058]
  This study aims to extend the framework for assessing artificial intelligence, called GROW-AI.<n>GROW-AI is designed to answer the question "Can machines grow up?" -- a natural successor to the Turing Test.<n>The originality of the work lies in the conceptual transposition of the process of "growing" from the human world to that of artificial intelligence.
  arXiv  Detail & Related papers  (2025-08-22T10:19:42Z)
• Machine vs Machine: Using AI to Tackle Generative AI Threats in Assessment [0.0]
  This paper presents a theoretical framework for addressing the challenges posed by generative artificial intelligence (AI) in higher education assessment.<n>Large language models like GPT-4, Claude, and Llama increasingly demonstrate the ability to produce sophisticated academic content.<n>Surveys indicate 74-92% of students experimenting with these tools for academic purposes.
  arXiv  Detail & Related papers  (2025-05-31T22:29:43Z)
• Two Experts Are All You Need for Steering Thinking: Reinforcing Cognitive Effort in MoE Reasoning Models Without Additional Training [86.70255651945602]
  We introduce a novel inference-time steering methodology called Reinforcing Cognitive Experts (RICE)<n>RICE aims to improve reasoning performance without additional training or complexs.<n> Empirical evaluations with leading MoE-based LRMs demonstrate noticeable and consistent improvements in reasoning accuracy, cognitive efficiency, and cross-domain generalization.
  arXiv  Detail & Related papers  (2025-05-20T17:59:16Z)
• How Metacognitive Architectures Remember Their Own Thoughts: A Systematic Review [16.35521789216079]
  Metacognition has gained significant attention for its potential to enhance autonomy and adaptability of artificial agents.<n>Existing overviews remain at a conceptual level that is undiscerning to the underlying algorithms, representations, and their respective success.
  arXiv  Detail & Related papers  (2025-02-28T08:48:41Z)
• A Unified Framework for Evaluating the Effectiveness and Enhancing the Transparency of Explainable AI Methods in Real-World Applications [2.0681376988193843]
  This study introduces a single evaluation framework for XAI.<n>It uses both numbers and user feedback to check if the explanations are correct, easy to understand, fair, complete, and reliable.<n>We show the value of this framework through case studies in healthcare, finance, farming, and self-driving systems.
  arXiv  Detail & Related papers  (2024-12-05T05:30:10Z)
• Neural Causal Models for Counterfactual Identification and Estimation [62.30444687707919]
  We study the evaluation of counterfactual statements through neural models. First, we show that neural causal models (NCMs) are expressive enough. Second, we develop an algorithm for simultaneously identifying and estimating counterfactual distributions.
  arXiv  Detail & Related papers  (2022-09-30T18:29:09Z)
• Neuro-symbolic Architectures for Context Understanding [59.899606495602406]
  We propose the use of hybrid AI methodology as a framework for combining the strengths of data-driven and knowledge-driven approaches. Specifically, we inherit the concept of neuro-symbolism as a way of using knowledge-bases to guide the learning progress of deep neural networks.
  arXiv  Detail & Related papers  (2020-03-09T15:04:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.