From Knowing to Acting: Benchmarking Self-Awareness Capability of LLM Agents

• URL: http://arxiv.org/abs/2606.20661v1
• Date: Tue, 09 Jun 2026 17:17:08 GMT
• Title: From Knowing to Acting: Benchmarking Self-Awareness Capability of LLM Agents
• Abstract summary: We introduce KAPRO, a framework that evaluates cognitive-behavioral alignment by decoupling an agent's metacognitive judgment (Knowing) from its spontaneous execution (Acting)<n>Experiments show that self-awareness capability is strongly correlated with task success but degrades sharply in internal-capability settings.<n> proprietary and reasoning-oriented models demonstrate more reliable cognitive gating.
• Score: 69.07911499189129
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The integration of external tools has transitioned LLM agents from passive responders to autonomous systems. However, current benchmarks prioritize execution success, neglecting self-awareness capability, the ability to discern whether a problem requires necessary external resources or can be solved via internal parametric knowledge. To address this, we introduce KAPRO (Knowing-Acting Quadrant PRObe), a framework that evaluates cognitive-behavioral alignment by decoupling an agent's metacognitive judgment (Knowing) from its spontaneous execution (Acting). We further construct KAware, a dataset rigorously partitioning tasks into external, internal, and hybrid subspaces to systematically probe these epistemic boundaries. Extensive experiments across diverse agent architectures show that self-awareness capability is strongly correlated with task success but degrades sharply in internal-capability settings. Moreover, open-source and instruction-following models exhibit stronger tool overuse due to shallow pattern matching, while proprietary and reasoning-oriented models demonstrate more reliable cognitive gating. Benchmark and codes are available at https://github.com/AI-Santiago/KAware.

Related papers

• SAEScientist-Bench: Can AI Agents Conduct Autonomous SAE Interpretability Research? [26.940600916534446]
  We introduce SAEScientist-Bench to evaluate whether AI agents can act as scientists utilizing SAE tools for autonomous mechanistic discovery.<n>Given a target concept, an agent designs contrastive probes and navigates a Gemma Scope dictionary of 131K+ features to discover the optimal feature.<n>Across 10 agent configurations and 20 tasks, frontier agents demonstrate genuine discovery capabilities but remain well behind the expert baseline.
  arXiv  Detail & Related papers  (2026-09-08T17:45:09Z)
• From Sequence to Structure: Relational Uncertainty Propagation for LLM Agents [75.69180947909148]
  We propose a trajectory-level UQ framework for large language model (LLM) agents.<n>RuPA represents an execution history as a directed trajectory graph in which reasoning states, tool interactions, and environment feedback are nodes connected by temporal and semantic dependency edges.<n>We evaluate RUPA on representative agent benchmarks, including $$-2, Terminal-Bench-2, and GAIA, using 6 open-source LLMs spanning multiple model families.
  arXiv  Detail & Related papers  (2026-08-17T01:40:14Z)
• Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skills [53.15216425100295]
  We introduce Skill Self-Play (Skill-SP), a co-evolutionary framework comprising a proposer, a solver, and a skill controller.<n>The proposer generates challenging tasks conditioned on dynamically sampled skills; the solver explores candidate solutions to push its capability boundaries; and the skill controller collects execution feedback to update and expand the skill library.<n>This interactive co-evolution effectively bridges the gap between structured verification and open-ended exploration.
  arXiv  Detail & Related papers  (2026-07-24T17:59:22Z)
• SAKE: Self-aware Knowledge Exploitation-Exploration for Grounded Multimodal Named Entity Recognition [28.17858615204594]
  Grounded Multimodal Named Entity Recognition (GMNER) aims to extract named entities and localize their visual regions within image-text pairs.<n>In open-world social media platforms, GMNER remains challenging due to the prevalence of long-tailed, rapidly evolving, and unseen entities.<n>We propose SAKE, an end-to-end agentic framework that harmonizes internal knowledge exploitation and external knowledge exploration.
  arXiv  Detail & Related papers  (2026-04-22T03:17:36Z)
• The Cognitive Circuit Breaker: A Systems Engineering Framework for Intrinsic AI Reliability [0.0]
  Large Language Models (LLMs) are increasingly deployed in mission-critical software systems.<n>We propose the Cognitive Circuit Breaker, a novel systems engineering framework that provides intrinsic reliability monitoring with minimal latency overhead.<n>We demonstrate statistically significant detection of cognitive dissonance, highlight architecture-dependent Out-of-Distribution (OOD) generalization, and show that this framework adds negligible computational overhead to the active inference pipeline.
  arXiv  Detail & Related papers  (2026-04-15T02:34:37Z)
• Why Cognitive Robotics Matters: Lessons from OntoAgent and LLM Deployment in HARMONIC for Safety-Critical Robot Teaming [2.779709069421631]
  We present HARMONIC, a cognitive-robotic architecture that pairs OntoAgent, a content-centric cognitive architecture, with a tactical reactive layer.<n>Results reveal that LLMs do not consistently assess their own knowledge state before acting, causing downstream failures in diagnostic reasoning and action selection.<n>These findings support the design of physically embodied systems in which cognitive architectures retain primary authority for reasoning.
  arXiv  Detail & Related papers  (2026-03-20T09:31:36Z)
• Improving Chain-of-Thought for Logical Reasoning via Attention-Aware Intervention [4.584629831500306]
  A non-interactive, end-to-end framework enables reasoning to emerge within the model itself.<n>We show that introducing structural information into the few-shot prompt activates a subset of attention heads that patterns aligned with logical reasoning operators.<n>We propose Attention-Aware Intervention (AAI), an inference-time intervention method that reweights attention scores across selected heads identified by their logical patterns.
  arXiv  Detail & Related papers  (2026-01-14T19:10:10Z)
• Agentic Knowledgeable Self-awareness [79.25908923383776]
  KnowSelf is a data-centric approach that applies agents with knowledgeable self-awareness like humans.<n>Our experiments demonstrate that KnowSelf can outperform various strong baselines on different tasks and models with minimal use of external knowledge.
  arXiv  Detail & Related papers  (2025-04-04T16:03:38Z)
• Interactive Agents to Overcome Ambiguity in Software Engineering [61.40183840499932]
  AI agents are increasingly being deployed to automate tasks, often based on ambiguous and underspecified user instructions.<n>Making unwarranted assumptions and failing to ask clarifying questions can lead to suboptimal outcomes.<n>We study the ability of LLM agents to handle ambiguous instructions in interactive code generation settings by evaluating proprietary and open-weight models on their performance.
  arXiv  Detail & Related papers  (2025-02-18T17:12:26Z)
• Adaptive Tool Use in Large Language Models with Meta-Cognition Trigger [49.81945268343162]
  We propose MeCo, an adaptive decision-making strategy for external tool use.<n>MeCo quantifies metacognitive scores by capturing high-level cognitive signals in the representation space.<n>MeCo is fine-tuning-free and incurs minimal cost.
  arXiv  Detail & Related papers  (2025-02-18T15:45:01Z)
• RCAgent: Cloud Root Cause Analysis by Autonomous Agents with Tool-Augmented Large Language Models [46.476439550746136]
  Large language model (LLM) applications in cloud root cause analysis (RCA) have been actively explored recently. We present RCAgent, a tool-augmented LLM autonomous agent framework for practical and privacy-aware industrial RCA usage. Running on an internally deployed model rather than GPT families, RCAgent is capable of free-form data collection and comprehensive analysis with tools.
  arXiv  Detail & Related papers  (2023-10-25T03:53:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.