Cognitive Fatigue in Autoregressive Transformers: Formalization and Measurement

• URL: http://arxiv.org/abs/2605.30981v1
• Date: Fri, 29 May 2026 08:18:23 GMT
• Title: Cognitive Fatigue in Autoregressive Transformers: Formalization and Measurement
• Abstract summary: We formalize cognitive fatigue as a measurable generation-time state characterized by decay in attention to the original prompt, representational drift, and entropy miscalibration.<n>We introduce the Fatigue Index (FI), a model-agnostic diagnostic that aggregates these three signals under explicit axioms.<n>These results establish cognitive fatigue as a coherent and measurable phenomenon, and position FI as a tool for runtime reliability monitoring.
• Score: 2.709736780477742
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Autoregressive language models frequently degrade during long-horizon generation, producing repetitive text, losing instruction adherence, and exhibiting unstable entropy. Despite the prevalence of these failures, practitioners lack online diagnostics to detect them in real-time as they occur. We formalize this degradation as cognitive fatigue, a measurable generation-time state characterized by decay in attention to the original prompt, representational drift, and entropy miscalibration. We introduce the Fatigue Index (FI), a lightweight, model-agnostic diagnostic that aggregates these three signals under explicit axioms (monotonicity, boundedness, interpretability) enabling reliable runtime monitoring. Across nine models (1B-13B parameters), FI trajectories exhibit structured temporal dynamics, predict task degradation (AUROC = 0.95) and repetition (Spearman rho = 0.94), and reveal non-monotonic scaling behavior: instruction-tuned models below 3B exhibit faster collapse than base models, with this trend reversing at 7B. Stress analyses further show that FI onset accelerates under longer contexts, middle-positioned evidence, and reduced numerical precision. These results establish cognitive fatigue as a coherent and measurable phenomenon, and position FI as a principled tool for runtime reliability monitoring in production LLM systems.

Related papers

• TTSD-FAR: Test-Time Self-Distillation with Fisher-Anchored Restoration for Missing-Modality Emotion Recognition in LVLMs [83.78708832901194]
  Large video-language models (LVLMs) have shown remarkable performance on multimodal tasks like multimodal emotion recognition (ER) in the wild.<n>However, real-world deployment remains challenging: modalities may be missing or noisy at test time.<n>We propose a solution in which a frozen teacher guides an adaptive low-rank student via self-distillation.<n>Experiments on MELD, DFEW, and BAH under 0%-50% missing modalities show that this unified adaptation-restoration design consistently outperforms entropy-based adaptation, RAG, and perplexity-based generation.
  arXiv  Detail & Related papers  (2026-08-18T23:40:28Z)
• Where to Refine, When to Stop: Rethinking Redundancy via Latent Discrepancy for Efficient Visual Autoregressive Generation [50.139984798361375]
  We propose a training-free framework that removes redundancy via latent discrepancy.<n> LD-Pruning substantially reduces inference while maintaining high generation quality, achieving up to 2.35x speedup on Infinity-8B.
  arXiv  Detail & Related papers  (2026-05-29T19:34:39Z)
• Stabilizing Temporal Inference Dynamics for Online Surgical Phase Recognition [42.76777398447746]
  Online Surgical Phase Recognition (SPR) models can reach high frame-wise accuracy, yet their predictions often lack temporal stability.<n>We show that this instability is not random noise but arises from two mechanisms: early misclassifications corrupt temporal feature states and propagate forward to form error cascades.<n>We propose a unified Train-Inference-Evaluation framework that explicitly stabilizes temporal inference dynamics using model-agnostic, plug-and-play components.
  arXiv  Detail & Related papers  (2026-05-11T16:56:15Z)
• A Progressive Training Strategy for Vision-Language Models to Counteract Spatio-Temporal Hallucinations in Embodied Reasoning [49.61652671596548]
  "Multi-image hallucination reasoning", where a massive performance drop between forward and temporal queries reveals a dependence on superficial shortcuts instead of genuine understanding.<n>To mitigate this, we develop a new dataset: Chain-of-temporal building on detailed reasoning into Chain steps and definitive judgments.<n>Our experiments demonstrate that this approach not only improves accuracy but also the forward-backward performance gap from over 70% to only 6.53%.
  arXiv  Detail & Related papers  (2026-04-12T07:48:44Z)
• The Stationarity Bias: Stratified Stress-Testing for Time-Series Imputation in Regulated Dynamical Systems [0.098314893665023]
  Time-series imputation benchmarks use random masking and shape-agnostic metrics.<n>We formalize this bias and propose a emphStratified Stress-Test that partitions evaluation into Stationary and Transient regimes.
  arXiv  Detail & Related papers  (2026-02-17T15:05:56Z)
• Information Fidelity in Tool-Using LLM Agents: A Martingale Analysis of the Model Context Protocol [69.11739400975445]
  We introduce the first theoretical framework for analyzing error accumulation in Model Context Protocol (MCP) agents.<n>We show that cumulative distortion exhibits linear growth and high-probability deviations bounded by $O(sqrtT)$.<n>Key findings include: semantic weighting reduces distortion by 80%, and periodic re-grounding approximately every 9 steps suffices for error control.
  arXiv  Detail & Related papers  (2026-02-10T21:08:53Z)
• "I May Not Have Articulated Myself Clearly": Diagnosing Dynamic Instability in LLM Reasoning at Inference Time [1.570719611178015]
  Reasoning failures in large language models (LLMs) are typically measured only at the end of a generation, yet many failures manifest as a process-level breakdown.<n>We define a simple instability signal that combines consecutive-step distributional shift (JSD) and uncertainty (entropy)<n>Across GSM8K and HotpotQA, instability strength predicts wrong answers with above-chance AUC and yields monotonic bucket-level accuracy decline.
  arXiv  Detail & Related papers  (2026-02-02T22:11:25Z)
• EDIS: Diagnosing LLM Reasoning via Entropy Dynamics [3.858418431840288]
  We show that the emphtemporal evolution of confidence during generation carries richer information than aggregate statistics alone.<n>We introduce the Entropy Dynamics Instability Score (textbfEDIS), a trajectory-level metric quantifying instability in entropy evolution.
  arXiv  Detail & Related papers  (2026-02-01T15:43:50Z)
• GO-OSC and VASH: Geometry-Aware Representation Learning for Early Degradation Detection in Oscillatory Systems [0.0]
  We introduce GO-OSC, a geometry-aware representation learning framework for oscillatory time series.<n>We show that under early phase-only degradation, energy-based statistics have zero first-order detection power, whereas geometric probes achieve strictly positive sensitivity.<n>Our analysis characterizes when and why linear probing fails under non-identifiable representations and shows how canonicalization restores statistical detectability.
  arXiv  Detail & Related papers  (2026-01-24T09:35:57Z)
• Extreme Miscalibration and the Illusion of Adversarial Robustness [66.29268991629085]
  Adversarial Training is often used to increase model robustness. We show that this observed gain in robustness is an illusion of robustness (IOR) We urge the NLP community to incorporate test-time temperature scaling into their robustness evaluations.
  arXiv  Detail & Related papers  (2024-02-27T13:49:12Z)
• Gradient Starvation: A Learning Proclivity in Neural Networks [97.02382916372594]
  Gradient Starvation arises when cross-entropy loss is minimized by capturing only a subset of features relevant for the task. This work provides a theoretical explanation for the emergence of such feature imbalance in neural networks.
  arXiv  Detail & Related papers  (2020-11-18T18:52:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.