Where Reliability Lives in Vision-Language Models: A Mechanistic Study of Attention, Hidden States, and Causal Circuits

• URL: http://arxiv.org/abs/2605.08200v1
• Date: Tue, 05 May 2026 22:27:05 GMT
• Title: Where Reliability Lives in Vision-Language Models: A Mechanistic Study of Attention, Hidden States, and Causal Circuits
• Abstract summary: A pervasive takeaway holds that legible-language models (VLMs) are most trustworthy when their attention maps sharp.<n>We compare attention structure, generation dynamics, and hidden-state geometry against a single correctness label.<n>In 3-7B VLMs, reliability is read more reliably off hidden-state geometry, layer-wise margin formation, and sparse late-layer circuits than off attention-map sharpness.
• Score: 1.840562129212051
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A pervasive intuition holds that vision-language models (VLMs) are most trustworthy when their attention maps look sharp: concentrated attention on the queried region should imply a confident, calibrated answer. We test this Attention-Confidence Assumption directly. We instrument three open-weight VLM families (LLaVA-1.5, PaliGemma, Qwen2-VL; 3-7B parameters) with a unified mechanistic pipeline -- the VLM Reliability Probe (VRP) -- that compares attention structure, generation dynamics, and hidden-state geometry against a single correctness label. Three results emerge. (i) Attention structure is a near-zero predictor of correctness (R_pb(C_k,y)=0.001, 95% CI [-0.034,0.036]; R_pb(H_s,y)=-0.012, [-0.047,0.024] on a pooled n=3,090 split), even though attention remains causally necessary for feature extraction (top-30% patch masking drops accuracy by 8.2-11.3 pp, p<0.001). (ii) Reliability becomes legible later in the computation: a single hidden-state linear probe reaches AUROC>0.95 on POPE for two of three families, and self-consistency at K=10 is the strongest behavioral predictor we measure at 10x inference cost (R_pb=0.43). (iii) Causal neuron-level ablations expose a sharp architectural split with direct monitor-design implications: late-fusion LLaVA concentrates reliability in a fragile late bottleneck (-8.3 pp object-identification accuracy after top-5 probe-neuron ablation), whereas early-fusion PaliGemma and Qwen2-VL distribute it widely and absorb destruction of ~50% of their peak-layer hidden dimension with <=1 pp degradation. The takeaway is narrow but consequential: in 3-7B VLMs, reliability is read more reliably off hidden-state geometry, layer-wise margin formation, and sparse late-layer circuits than off attention-map sharpness.

Related papers

• A Few Neurons Reveal When LLMs Misuse Tools: Sparse Detection and Selective Steering for Reliable Tool Use [28.57781785937644]
  We introduce PRISMS (Probing Representations In Support of Monitoring and Steering), a closed-loop framework that shares a failure-specific neuron basis between sparse detection and activation.<n>Across six models, PRISMS reduces pooled over-calling rate by 80% while increasing tool-required accuracy by 14.2 percentage points.
  arXiv  Detail & Related papers  (2026-07-31T19:03:44Z)
• Attention Degradation, Function Token Anchoring, and the Limits of Attention-Based Intervention in Large Language Models [0.0]
  We present six coordinated experiments across GPT-2, LLaMA-3.2-1B/3B, OPT-1.3B, and distilgpt2.<n>We first characterise short-term (5-100 token) attention degradation, finding a universal exponential-then-plateau pattern.<n>We conclude that mean attention degradation is largely rather than prescriptive.
  arXiv  Detail & Related papers  (2026-07-09T07:04:07Z)
• Geometric Collapse: When Vision Models Fail to Verify Physical Causality [52.43819179934414]
  We propose Scrambled Edges, a controlled counterfactual that injects salient edge-like cues.<n>With energy-matched and structure-matched controls, we isolate the effect of unsupported edge evidence from high-frequency energy and edge sparsity.<n>Across CNN/ViT/SSL depth predictors, Scrambled Edges induce up to 3.2x larger deviation from clean predictions than energy-matched noise.
  arXiv  Detail & Related papers  (2026-07-08T00:04:02Z)
• Visuals Lie, Consistency Speaks: Disentangling Spatial Attention from Reliability in Vision-Language Models [6.612737895321556]
  MultiSymbol Foundation Models are increasingly used as reasoning agents, making reliability, knowing when a model may hallucinate, critical.<n>A common intuition, which we call the Attention-Confidence Assumption, holds that reliability follows from "structural" visual perception.<n>We challenge this through the VLM Reliability Probe (VRP), a systematic cross-family study of reliability signals in contemporary Vision-Language Models (VLMs).
  arXiv  Detail & Related papers  (2026-06-16T00:58:43Z)
• Unveiling the Entropy Dynamics of Chain-of-Thought Reasoning [81.57028614960576]
  We show that the Confidence Region possesses two critical properties: 1) High Reliability -- answers in the confidence region become highly accurate and stable, and 2) High Redundancy -- models generate unnecessary tokens long after reaching the correct answer.<n>These properties unlock more efficient and reliable inference strategies.
  arXiv  Detail & Related papers  (2026-06-01T10:11:14Z)
• Alignment Collapse Under KV Cache Quantization: Diagnosis and Mitigation [6.129872931808218]
  Key-value ( KV) cache quantization is widely used to reduce Large Language Model (LLM) inference memory.<n>In this study, we explore alignment preservation under KV cache quantization.<n>We find that low-bit quantization can silently destroy safety alignment.
  arXiv  Detail & Related papers  (2026-06-01T02:02:20Z)
• SAFE-Pruner: Semantic Attention-Guided Future-Aware Token Pruning for Efficient Vision-Language-Action Manipulation [71.91583446135651]
  We propose SAFE-Pruner, a plug-and-play pruning framework that incorporates attention cues of future layers into pruning decisions.<n>We show that our method achieves up to 1.89x speedup with a minimal degradation in success rate of less than 1.7%, while outperforming state-of-the-art methods by up to 1.9%.
  arXiv  Detail & Related papers  (2026-05-28T09:23:08Z)
• Architectural Observability Collapse in Transformers [0.0]
  Training preserved an internal decision-quality signal that output confidence does not expose.<n>Confidence controls absorb on average 60.3% of raw probe signal across 14 models in 6 families.<n>Training preserved observability where Llama 3.1 8B collapses at identical 32-layer, 32-head, 4096-hidden shape.
  arXiv  Detail & Related papers  (2026-04-27T02:39:02Z)
• Uncertainty-Aware Mapping from 3D Keypoints to Anatomical Landmarks for Markerless Biomechanics [39.155383773210836]
  Markerless biomechanics increasingly relies on 3D skeletal keypoints extracted from video.<n>In this work, we investigate predictive uncertainty as a measure of confidence for mapping 3D pose keypoints to 3D anatomical landmarks.
  arXiv  Detail & Related papers  (2026-03-27T09:42:23Z)
• Severe Domain Shift in Skeleton-Based Action Recognition:A Study of Uncertainty Failure in Real-World Gym Environments [0.0]
  We present a systematic study of a severe domain shift using a novel Gym2D dataset and the UCF101 dataset.<n>Our work challenges standard deployment assumptions, providing a principled safety analysis of both semantic and geometric skeleton recognition deployment.
  arXiv  Detail & Related papers  (2026-03-16T17:37:17Z)
• Linear Predictability of Attention Heads in Large Language Models [0.5833117322405447]
  We show that pretrained Transformers exhibit a pervasive inter-head linear structure.<n>We exploit this by caching only reference-head KV states and reconstructing the remaining heads on the fly.
  arXiv  Detail & Related papers  (2026-03-04T09:04:03Z)
• BadCLIP++: Stealthy and Persistent Backdoors in Multimodal Contrastive Learning [73.46118996284888]
  Research on backdoor attacks against multimodal contrastive learning models faces two key challenges: stealthiness and persistence.<n>We propose BadCLIP++, a unified framework that tackles both challenges.<n>For stealthiness, we introduce a semantic-fusion QR micro-trigger that embeds imperceptible patterns near task-relevant regions.<n>For persistence, we stabilize trigger embeddings via radius shrinkage and centroid alignment.
  arXiv  Detail & Related papers  (2026-02-19T08:31:16Z)
• Information Fidelity in Tool-Using LLM Agents: A Martingale Analysis of the Model Context Protocol [69.11739400975445]
  We introduce the first theoretical framework for analyzing error accumulation in Model Context Protocol (MCP) agents.<n>We show that cumulative distortion exhibits linear growth and high-probability deviations bounded by $O(sqrtT)$.<n>Key findings include: semantic weighting reduces distortion by 80%, and periodic re-grounding approximately every 9 steps suffices for error control.
  arXiv  Detail & Related papers  (2026-02-10T21:08:53Z)
• Near-Oracle KV Selection via Pre-hoc Sparsity for Long-Context Inference [54.467557491325046]
  We propose Pre-hoc Sparsity (PrHS), which selects KV entries before attention scoring and provides explicit accuracy control.<n>PrHS reduces retrieval overhead by over 90%, achieving 3x higher retrieval sparsity than HShare at matched or better accuracy.<n>It incurs under 1% average degradation on LongBench, lowers attention FLOPs by about 15% versus prior baselines, and yields a 9.9x speedup in attention-operator latency and 2.8x higher throughput.
  arXiv  Detail & Related papers  (2026-02-09T07:05:23Z)
• Benchmarking Corruption Robustness of LVLMs: A Discriminative Benchmark and Robustness Alignment Metric [49.393713730706445]
  We introduce Bench-C, a benchmark emphasizing discriminative samples for assessing corruption robustness.<n>We propose the Robustness Alignment Score (RAS), a unified metric that measures degradation in logit-level prediction structure.
  arXiv  Detail & Related papers  (2025-11-24T12:07:56Z)
• Towards Adversarial Robustness and Uncertainty Quantification in DINOv2-based Few-Shot Anomaly Detection [6.288045889067255]
  Foundation models such as DINOv2 have shown strong performance in few-shot anomaly detection.<n>We present one of the first systematic studies of adversarial attacks and uncertainty estimation in this setting.<n>We find that raw anomaly scores are poorly calibrated, revealing a gap between confidence and correctness that limits safety-critical use.
  arXiv  Detail & Related papers  (2025-10-15T15:06:45Z)
• vAttention: Verified Sparse Attention [100.98210818821688]
  vAttention is a practical sparse attention mechanism with user-specified $(epsilon, delta)$ guarantees on approximation accuracy (thus, verified)<n>We show that vAttention significantly improves the quality of sparse attention across datasets.<n>It can be deployed in reasoning scenarios to achieve fast decoding without compromising model quality.
  arXiv  Detail & Related papers  (2025-10-07T08:46:08Z)
• Free Lunch for Generating Effective Outlier Supervision [46.37464572099351]
  We propose an ultra-effective method to generate near-realistic outlier supervision. Our proposed textttBayesAug significantly reduces the false positive rate over 12.50% compared with the previous schemes.
  arXiv  Detail & Related papers  (2023-01-17T01:46:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.