Beyond Accuracy: Auditing Spatial Provenance in Visual Token Pruning for OCR-Critical MLLM Inference

• URL: http://arxiv.org/abs/2608.00077v1
• Date: Wed, 29 Jul 2026 13:45:31 GMT
• Title: Beyond Accuracy: Auditing Spatial Provenance in Visual Token Pruning for OCR-Critical MLLM Inference
• Abstract summary: Visual-token pruning is judged by answer quality at a fixed retention budget.<n>We turn this blind spot into an evidence-risk audit that couples answer behavior with geometric token-origin provenance.<n>We show that favorable target-verification points do not imply task-general compression.
• Score: 22.335515110971475
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Visual-token pruning is usually judged by answer quality at a fixed retention budget. For text-rich multimodal large language models (MLLMs), this protocol can miss a distinct failure: an answer remains correct even when no retained token is locally traceable to the small OCR region that supports it. We turn this blind spot into an evidence-risk audit that couples answer behavior with geometric token-origin provenance, interventions, and realized cost; transparent training-free selectors isolate controlled operating points. On locked image-disjoint confirmation, Qwen Target at 30% retention has observed accuracy 0.786 versus 0.783 for Full (paired image-cluster difference +0.003, 95% CI [-0.014, +0.020]), yet same-budget Target, Random, and Grid retain sharply different positive-support coverage: 0.620, 0.270, and 0.318. Across Qwen3-VL-8B, LLaVA-1.5-7B, and InternVL3.5-8B, matched controls, interventions, detector tests, and external methods reveal model-specific quality-risk-traceability frontiers that accuracy alone does not expose. Materialized prefixes yield up to 4.32x batch-prefill speedup and 76.4% lower incremental peak memory; full-validation TextVQA and DocVQA further show that favorable target-verification points do not imply task-general compression. Visual-token pruning should therefore report surviving spatial provenance and realized cost alongside quality and compression.

Related papers

• RoRA: Role-Oriented Regional Allocation for Visual Token Pruning in MLLMs [50.93650157115467]
  We present RoRA, a training-free framework that casts visual token pruning as role-oriented regional evidence allocation.<n>Under matched budgets, RoRA consistently outperforms strong training-free baselines across LLaVA and Qwen-VL families.<n>At a 66.7% pruning ratio, RoRA requires only 0.7 ms for token selection and reduces end-to-end inference time by 24.6%.
  arXiv  Detail & Related papers  (2026-08-07T10:39:47Z)
• ET-Prune: Evidence-Aware Dynamic Budgeting for Visual Token Pruning in Text-Rich MLLMs [37.20525993006875]
  We present ET-Prune, a training-free framework that casts pruning as evidence allocation.<n>It derives question-conditioned evidence from a decoder-side partial query-key block.<n>It safeguards text-like spatial regions, and converts evidence uncertainty and density into a sample-specific token floor.
  arXiv  Detail & Related papers  (2026-08-03T09:42:34Z)
• Operational Proto-Introspection in Looped Language Models: Process-Quality Taps, Executable Branching, and the Readout-Control Boundary [0.0]
  Can a language model read the quality of its ongoing computation?<n>Can an external intervention turn that readout into better outcomes?<n>We test both questions in a frozen 2.6B looped transformer, Ouro-RLTT.
  arXiv  Detail & Related papers  (2026-07-20T22:40:36Z)
• Know Before You Fetch: Calibrated Retrieval-Budget Allocation for Retrieval-Augmented Generation [10.60293484295824]
  We formulate adaptive RAG as calibrated retrieval-budget allocation.<n>We calibrate sequence log-probability and prefix-logit uncertainty signals into probabilities of correctness.<n>For sequence log-probability, ECE drops from 0.275 to 0.062 on TriviaQA, 0.643 to 0.009 on NQ, and 0.711 to 0.031 on MS MARCO.
  arXiv  Detail & Related papers  (2026-06-29T08:36:33Z)
• Beyond Logprobs: A Multi-Signal Confidence Engine for LLM-Based Document Field Extraction [2.1997815944423516]
  ExtractConf is a cross-domain, field-agnostic confidence engine.<n>It grounds confidence estimation in two structurally different readings of the same document.<n>It achieves 0.928 ROC AUC and reduces selective prediction risk by 70% over logprob-mean.
  arXiv  Detail & Related papers  (2026-06-23T10:58:08Z)
• Amplifying, Not Learning: Fine-Tuned AI Text Detectors Amplify a Pretrained Direction [51.56484100374058]
  Text detectors amplify a pretrained typicality axis; they do not construct an AI-vs-human boundary.<n>On raw encoders before any task supervision, projecting onto centroid(AI)-centroid(HC3) achieves NYT-vs-HC3 AUROC 0.806/0.944/0.834 across three architectures.<n>On RoBERTa-base, raw projection exceeds fine-tuning; on RoBERTa-base, full fine-tuning reduces discrimination below raw on both fluent-formal populations tested.
  arXiv  Detail & Related papers  (2026-05-20T19:08:38Z)
• Distributional Energy-Based Models for Uncertainty-Aware Structured LLM Reasoning [40.342912574072024]
  Large Language Models produce structured outputs such as travel plans or code solutions.<n>Individual reasoning steps may appear correct while the output as a whole violates budgets, fails test cases, or contradicts earlier deductions.<n>We propose a learned quality scorer with deterministic analytical constraint penalties for verifying structured LLM outputs.
  arXiv  Detail & Related papers  (2026-05-15T17:08:27Z)
• Between the Layers Lies the Truth: Uncertainty Estimation in LLMs Using Intra-Layer Local Information Scores [36.86032736109853]
  Large language models (LLMs) are often confidently wrong, making reliable uncertainty estimation (UE) essential.<n>We propose a compact, per-instance UE method that scores cross-layer patterns in internal representations using a single forward pass.
  arXiv  Detail & Related papers  (2026-03-17T08:35:14Z)
• From Black Box to Glass Box: Cross-Model ASR Disagreement to Prioto Review in Ambient AI Scribe Documentation [43.148402136307716]
  Cross-model disagreement among heterogeneous ASR systems can act as a reference-free uncertainty signal.<n>We transcribed 50 publicly available medical audio clips with eight ASR systems spanning commercial APIs and open-source engines.<n>Low-agreement regions were enriched for content disagreements, with the content fraction increasing from 53.9% to 73.9% across quintiles of high-risk mass.
  arXiv  Detail & Related papers  (2026-03-02T13:02:13Z)
• CARE What Fails: Contrastive Anchored-REflection for Verifiable Multimodal [84.71254539482369]
  Group-relative reinforcement learning with verifiable rewards (RLVR) often wastes the most informative data it already has the failures.<n>We present CARE, a failure-centric post-training framework for multimodal reasoning that turns errors into supervision.<n> CARE improves accuracy and training smoothness while explicitly increasing the share of learning signal that comes from failures.
  arXiv  Detail & Related papers  (2025-12-22T16:34:21Z)
• Evaluating Reasoning Faithfulness in Medical Vision-Language Models using Multimodal Perturbations [19.488236277427358]
  Vision-language models (VLMs) often produce chain-of-thought (CoT) explanations that sound plausible yet fail to reflect the underlying decision process.<n>We present a clinically grounded framework for chest X-ray visual question answering (VQA) that probes CoT faithfulness via controlled text and image modifications.
  arXiv  Detail & Related papers  (2025-10-13T09:28:22Z)
• vAttention: Verified Sparse Attention [100.98210818821688]
  vAttention is a practical sparse attention mechanism with user-specified $(epsilon, delta)$ guarantees on approximation accuracy (thus, verified)<n>We show that vAttention significantly improves the quality of sparse attention across datasets.<n>It can be deployed in reasoning scenarios to achieve fast decoding without compromising model quality.
  arXiv  Detail & Related papers  (2025-10-07T08:46:08Z)
• Unsupervised Conformal Inference: Bootstrapping and Alignment to Control LLM Uncertainty [49.19257648205146]
  We propose an unsupervised conformal inference framework for generation.<n>Our gates achieve close-to-nominal coverage and provide tighter, more stable thresholds than split UCP.<n>The result is a label-free, API-compatible gate for test-time filtering.
  arXiv  Detail & Related papers  (2025-09-26T23:40:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.