HyperLens: Quantifying Cognitive Effort in LLMs with Fine-grained Confidence Trajectory

• URL: http://arxiv.org/abs/2605.05741v1
• Date: Thu, 07 May 2026 06:32:57 GMT
• Title: HyperLens: Quantifying Cognitive Effort in LLMs with Fine-grained Confidence Trajectory
• Abstract summary: We identify an intrinsic magnification mechanism in transformer architectures.<n>We introduce HyperLens, a high-resolution probe designed to trace confidence trajectories.<n>Across datasets, HyperLens reveals a consistent divergence in confidence trajectories that separates complex from simple tasks.
• Score: 10.351595331564353
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While Large Language Models (LLMs) achieve strong performance across diverse tasks, their inference dynamics remain poorly understood because of the limited resolution of existing analysis tools. In this work, we identify an intrinsic magnification mechanism in transformer architectures: deeper layers inherently magnify the small changes of layer-wise confidence, providing a fine-grained confidence trajectory. Building on this insight, we introduce HyperLens, a high-resolution probe designed to trace confidence trajectories and quantify the cognitive effort during inference. Across LLMs and datasets, HyperLens reveals a consistent divergence in confidence trajectories that separates complex from simple tasks. We abstract this pattern into a quantitative cognitive effort metric. Our analysis reveals a fundamental principle: complex tasks consistently require higher cognitive effort. Finally, we provide a mechanistic diagnosis of a common side effect of standard Supervised Fine-Tuning (SFT): it can reduce cognitive effort and consequently degrade performance on in-domain tasks.

Related papers

• Reinforcement Learning with Metacognitive Feedback Elicits Faithful Uncertainty Expression in LLMs [60.80435138270317]
  We apply reinforcement learning with metacognitive feedback to the problem of faithful calibration.<n>Experiments show RLMF achieves generalizable, state-of-the-art FC on diverse tasks.<n> RLMF surpasses standard RL by up to 63% while enhancing models' ability to assess and express their own capability limits.
  arXiv  Detail & Related papers  (2026-06-30T17:56:01Z)
• Ensuring Logic in the Fog: Sound POMDP Synthesis with LTL Objectives [9.094127664014627]
  Linear Temporal Logic (LTL) provides a rigorous language for specifying such tasks.<n>In this paper, we bridge this gap with a novel, sound reward-shaping mechanism.<n>By integrating this mechanism into an enhanced Monte Carlo Planning framework, we empower agents to navigate the fog' of partial observability.
  arXiv  Detail & Related papers  (2026-05-12T15:46:35Z)
• Guided Collaboration in Heterogeneous LLM-Based Multi-Agent Systems via Entropy-Based Understanding Assessment and Experience Retrieval [35.96356869281219]
  We describe a counterintuitive phenomenon in the strong-weak system: a strong-weak collaboration may under-perform weak-weak combinations.<n>We propose an Entropy-Based Adaptive Guidance Framework that dynamically aligns the guidance with the cognitive state of each agent.<n>Our approach consistently enhances the effectiveness and stability of heterogeneous collaboration.
  arXiv  Detail & Related papers  (2026-02-14T07:10:04Z)
• ResMAS: Resilience Optimization in LLM-based Multi-agent Systems [37.355345383912756]
  Large Language Model-based Multi-Agent Systems (LLM-based MAS)<n>LLM-based MAS are typically distributed across different devices or environments, making them vulnerable to perturbations such as agent failures.<n>We study the resilience of MAS under perturbations and find that both the communication topology and prompt design significantly influence system resilience.
  arXiv  Detail & Related papers  (2026-01-08T08:03:37Z)
• Cognitive Mirrors: Exploring the Diverse Functional Roles of Attention Heads in LLM Reasoning [54.12174882424842]
  Large language models (LLMs) have achieved state-of-the-art performance in a variety of tasks, but remain largely opaque in terms of their internal mechanisms.<n>We propose a novel interpretability framework to systematically analyze the roles and behaviors of attention heads.
  arXiv  Detail & Related papers  (2025-12-03T10:24:34Z)
• Rethinking the Reliability of Multi-agent System: A Perspective from Byzantine Fault Tolerance [16.514747521376915]
  Large language models (LLMs) have established LLM-based agents as a major branch of multi-agent systems (MAS)<n>In this work, we investigate and quantify the reliability of LLM-based agents from the perspective of Byzantine fault tolerance.<n>We design CP-WBFT, a confidence probe-based weighted Byzantine Fault Tolerant consensus mechanism to enhance the stability of MAS with different topologies.
  arXiv  Detail & Related papers  (2025-11-13T15:20:12Z)
• Training-free Uncertainty Guidance for Complex Visual Tasks with MLLMs [61.64185573373394]
  We propose a training-free framework that uses an MLLM's intrinsic uncertainty as a proactive guidance signal.<n>We introduce a unified mechanism that scores candidate visual inputs by response uncertainty, enabling the model to autonomously focus on the most salient data.<n>Our work validates that harnessing intrinsic uncertainty is a powerful, general strategy for enhancing fine-grained multimodal performance.
  arXiv  Detail & Related papers  (2025-10-01T09:20:51Z)
• A Fano-Style Accuracy Upper Bound for LLM Single-Pass Reasoning in Multi-Hop QA [65.38186593873313]
  Multi-Hop Question Answering (MHQA) requires integrating dispersed, interdependent evidence through sequential reasoning under noise.<n>We introduce a proof-of-concept multi-call framework for MHQA, InfoQA.<n>We construct a stringent and noise-rich benchmark to validate our theory and framework.
  arXiv  Detail & Related papers  (2025-09-25T14:11:57Z)
• United Minds or Isolated Agents? Exploring Coordination of LLMs under Cognitive Load Theory [27.886952948525476]
  Large Language Models (LLMs) exhibit a notable performance ceiling on complex, multi-faceted tasks.<n>CoThinker is a novel framework designed to mitigate cognitive overload and enhance collaborative problem-solving abilities.
  arXiv  Detail & Related papers  (2025-06-07T15:48:04Z)
• Cognitive LLMs: Towards Integrating Cognitive Architectures and Large Language Models for Manufacturing Decision-making [51.737762570776006]
  LLM-ACTR is a novel neuro-symbolic architecture that provides human-aligned and versatile decision-making. Our framework extracts and embeds knowledge of ACT-R's internal decision-making process as latent neural representations. Our experiments on novel Design for Manufacturing tasks show both improved task performance as well as improved grounded decision-making capability.
  arXiv  Detail & Related papers  (2024-08-17T11:49:53Z)
• Corex: Pushing the Boundaries of Complex Reasoning through Multi-Model Collaboration [83.4031923134958]
  Corex is a suite of novel general-purpose strategies that transform Large Language Models into autonomous agents. Inspired by human behaviors, Corex is constituted by diverse collaboration paradigms including Debate, Review, and Retrieve modes. We demonstrate that orchestrating multiple LLMs to work in concert yields substantially better performance compared to existing methods.
  arXiv  Detail & Related papers  (2023-09-30T07:11:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.