LLMs Know When They Know, but Do Not Act on It: A Metacognitive Harness for Test-time Scaling

• URL: http://arxiv.org/abs/2605.14186v1
• Date: Wed, 13 May 2026 23:09:25 GMT
• Title: LLMs Know When They Know, but Do Not Act on It: A Metacognitive Harness for Test-time Scaling
• Abstract summary: We ask whether large language models (LLMs) possess latent metacognitive ability that can be turned into effective test-time control.<n>Inspired by the Nelson--Narens theory from cognitive psychology, we propose a metacognitive harness that separates monitoring from reasoning.
• Score: 26.999207995495354
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) often expose useful signals of self-monitoring: before solving a problem, they can estimate whether they are likely to succeed, and after solving it, they can judge whether their answer is likely to be correct. However, these signals are typically measured or elicited in isolation, rather than used to control inference. In this work, we ask whether LLMs possess latent metacognitive ability that can be turned into effective test-time control. Inspired by the Nelson--Narens theory from cognitive psychology, we propose a metacognitive harness that separates monitoring from reasoning. For each problem, the model first reports a pre-solve feeling-of-knowing (FOK) signal; after each solve attempt, it reports a post-solve judgment-of-learning (JOL) signal. Rather than treating these signals as passive confidence estimates, the harness turns them into an explicit control interface for reasoning: it decides when to trust the current solution, when to retry with compact metacognitive feedback, and when to pass multiple attempts to a final aggregator. Across text, code, and multimodal reasoning benchmarks, our harness substantially improves a fixed Claude Sonnet-4.6 base model without parameter updates or benchmark-specific fine-tuning. On the evaluated public benchmark snapshots, it raises pooled accuracy from 48.3 to 56.9 and exceeds the strongest listed leaderboard entries on the three primary evaluation settings: HLE-Verified, LiveCodeBench v6, and R-Bench-V. These results suggest that strong LLMs may already possess useful metacognitive ability, but require an explicit control harness to act on it during reasoning.

Related papers

• SAVOR: Self-Aware Visual Grounding via Confidence-Calibrated Reinforcement Learning for Multimodal Hallucination Mitigation [4.100479369058624]
  Multimodal large language models (MLLMs) produce fluent claims about objects, attributes, or relations that are not grounded in the image.<n>We introduce Savor, a training framework that augments the output schema with token and answer confidence.<n>We show that Savor reduces hallucination while preserving general capability on MME and MMBench, with lower Expected Error than DPO and decoding baselines.
  arXiv  Detail & Related papers  (2026-09-15T03:53:32Z)
• No Universal Signal Predicts Sample-Level LLM Regression under Version Updates [2.6426981014417055]
  This paper studies how to predict such regressions from signals available at inference time.<n> Practitioners can use these task-level patterns to choose which regression signal to trust for a given update.
  arXiv  Detail & Related papers  (2026-08-11T15:13:12Z)
• Enhancing LLM Metacognition via Cognitive Pairwise Training [56.19495984013656]
  Reinforcement learning with verifiable rewards can make models more willing to give confident answers when evidence or reasoning is unreliable.<n>Existing SFT or RL methods mainly teach LLMs to refuse or express uncertainty at the response level.<n>We propose Cognitive Pairwise Training (CPT), a cognitive mid-training alignment stage that turns pairwise comparisons over reasoning traces into a reusable alignment signal.
  arXiv  Detail & Related papers  (2026-05-30T19:53:19Z)
• Confidence-Orchestrated Self-Evolution against Uncertain LLM Feedback [8.780008955074967]
  Self-evolving large language models (LLMs) learn by generating their own training tasks and solutions, reducing reliance on human-curated supervision.<n>We propose COSE (Confidence-Orchestrated Self-Evolution), which uses the LLM's intrinsic confidence as a lightweight uncertainty signal to modulate learning.
  arXiv  Detail & Related papers  (2026-05-27T06:07:10Z)
• Understanding and Mitigating Premature Confidence for Better LLM Reasoning [76.16007941549857]
  Long chains of thought (CoT) from current language models frequently contain logical gaps and unjustified leaps.<n>We find such a signal in how the model's confidence evolves during reasoning.<n>We exploit this in progressive confidence shaping, a reinforcement learning objective that trains models to update their confidence as they reason rather than commit early.
  arXiv  Detail & Related papers  (2026-05-23T04:42:45Z)
• How LLMs Detect and Correct Their Own Errors: The Role of Internal Confidence Signals [6.467495925520036]
  Large language models can detect their own errors and sometimes correct them without external feedback.<n>We investigate this through the lens of second-order models of confidence from decision neuroscience.
  arXiv  Detail & Related papers  (2026-04-24T06:33:32Z)
• The Metacognitive Monitoring Battery: A Cross-Domain Benchmark for LLM Self-Monitoring [0.0]
  We introduce a cross-domain behavioural assay of monitoring-control coupling in LLMs, grounded in the Nelson and Narens (1990) metacognitive framework.<n>The battery-dependent calibration comprises monotonically decreasing (Q), monotonically increasing (GPT-5.4), or flat (Gemma)<n>Findings converge structurally with an independent Type-2 SDT approach, providing preliminary cross-method calibration.
  arXiv  Detail & Related papers  (2026-04-17T05:15:15Z)
• Reading Between the Lines: Abstaining from VLM-Generated OCR Errors via Latent Representation Probes [79.36545159724703]
  We propose Latent Representation Probing (LRP) to train lightweight probes on hidden states or attention patterns.<n>LRP improves abstention accuracy by 7.6% over best baselines.<n>This establishes a principled framework for building deployment-ready AI systems.
  arXiv  Detail & Related papers  (2025-11-25T00:24:42Z)
• Multi-Agent Evolve: LLM Self-Improve through Co-evolution [53.00458074754831]
  Reinforcement Learning (RL) has demonstrated significant potential in enhancing the reasoning capabilities of large language models (LLMs)<n>Recent Self-Play RL methods, inspired by the success of the paradigm in games and Go, aim to enhance LLM reasoning capabilities without human-annotated data.<n>We propose Multi-Agent Evolve (MAE), a framework that enables LLMs to self-evolve in solving diverse tasks, including mathematics, reasoning, and general knowledge Q&A.
  arXiv  Detail & Related papers  (2025-10-27T17:58:02Z)
• Critique to Verify: Accurate and Honest Test-Time Scaling with RL-Trained Verifiers [63.99316853136304]
  Mirror-Critique is a framework that trains a verifier with informative critiques.<n>We deploy a small instruction-tuned model to synthesize high-quality critique data.<n>The resulting Mirror-Verifier is deployed to evaluate candidate solutions by generating multiple critiques per solution.
  arXiv  Detail & Related papers  (2025-09-27T06:50:24Z)
• LACIE: Listener-Aware Finetuning for Confidence Calibration in Large Language Models [69.68379406317682]
  We introduce a listener-aware finetuning method (LACIE) to calibrate implicit and explicit confidence markers. We show that LACIE models the listener, considering not only whether an answer is right, but whether it will be accepted by a listener. We find that training with LACIE results in 47% fewer incorrect answers being accepted while maintaining the same level of acceptance for correct answers.
  arXiv  Detail & Related papers  (2024-05-31T17:16:38Z)
• SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step Reasoning [55.76083560152823]
  SelfCheck is a general-purpose zero-shot verification schema for recognizing errors in step-by-step reasoning. We test SelfCheck on three datasets (GSM8K, MathQA, and MATH) and find that it successfully recognizes errors and, in turn, increases final answer accuracies.
  arXiv  Detail & Related papers  (2023-08-01T10:31:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.