Index Light, Reason Deep: Deferred Visual Ingestion for Visual-Dense Document Question Answering

• URL: http://arxiv.org/abs/2602.14162v1
• Date: Sun, 15 Feb 2026 14:23:50 GMT
• Title: Index Light, Reason Deep: Deferred Visual Ingestion for Visual-Dense Document Question Answering
• Abstract summary: This paper proposes a demand-side ingestion strategy for multimodal document question answering.<n>It achieves comparable overall accuracy at zero ingestion VLM cost.<n>It also supports interactive refinement and progressive caching, transforming the "QA accuracy" problem into a "page localization" problem.
• Score: 3.0028612998967894
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing multimodal document question answering methods universally adopt a supply-side ingestion strategy: running a Vision-Language Model (VLM) on every page during indexing to generate comprehensive descriptions, then answering questions through text retrieval. However, this "pre-ingestion" approach is costly (a 113-page engineering drawing package requires approximately 80,000 VLM tokens), end-to-end unreliable (VLM outputs may fail to be correctly retrieved due to format mismatches in the retrieval infrastructure), and irrecoverable once it fails. This paper proposes the Deferred Visual Ingestion (DVI) framework, adopting a demand-side ingestion strategy: the indexing phase performs only lightweight metadata extraction, deferring visual understanding to the moment users pose specific questions. DVI's core principle is "Index for locating, not understanding"--achieving page localization through structured metadata indexes and BM25 full-text search, then sending original images along with specific questions to a VLM for targeted analysis. Experiments on two real industrial engineering drawings (113 pages + 7 pages) demonstrate that DVI achieves comparable overall accuracy at zero ingestion VLM cost (46.7% vs. 48.9%), an effectiveness rate of 50% on visually necessary queries (vs. 0% for pre-ingestion), and 100% page localization (98% search space compression). DVI also supports interactive refinement and progressive caching, transforming the "QA accuracy" problem into a "page localization" problem--once the correct drawing page is found, obtaining the answer becomes a matter of interaction rounds.

Related papers

• When Prompts Become Pixels: Prompt-Region Grounding for Multimodal Reasoning [73.56343820592399]
  We introduce Visualized Task Semantics (VTS), a controlled intervention that moves the question into the image while keeping the source problem and answer fixed.<n>Across six MLLMs and four benchmarks, accuracy drops in all 24 model-task pairs, by 17.8 points on average.<n>We present prompt-region grounding, whose core design aligns the question region with typed semantics and recovers its clean representation from a masked view.
  arXiv  Detail & Related papers  (2026-08-05T11:46:15Z)
• MAViE: A Multi-scale Adaptive Vision Encoder for Fine-grained Visual Perception and Efficient Multimodal Reasoning [0.0]
  method fuses shallow, intermediate, and deep features from a vision Transformer to preserve global semantics.<n>It then performs question-conditioned token routing according to question relevance, local information content, global semantics, and spatial coverage.<n>It reduces the average number of SigLIP-SO400M visual tokens from 729 to 146 (approximately 80.0%) and improves the mean score on VQAv2, GQA, TextVQA, ScienceQA-IMG, and MMBench by 2.2 percentage points.
  arXiv  Detail & Related papers  (2026-07-27T13:36:08Z)
• SAFE-Cascade: Cost-Adaptive Vision-Language Routing for Chart Question Answering [16.639536340934715]
  We demonstrate SAFE- Cascade, an interactive system for cost-adaptive chart question answering.<n>Given a chart image and a natural-language question, SAFE- Cascade first extracts chart text with OCR.<n>It obtains a provisional answer from a text-only language model, and then uses a learned router to decide whether to accept the text answer or escalate to a VLM.
  arXiv  Detail & Related papers  (2026-06-17T23:00:17Z)
• Detect Before You Leap: Mirage Detection in Vision-Language Models [2.6212127510234797]
  Vision-language models (VLMs) can produce confident visual answers even when the required visual evidence is missing, blank, or unrelated to the question.<n>We study pre-release mirage detection: given an image-question pair, the goal is to determine whether a VLM should answer or abstain.<n>We propose Text-Conditioned Layer-wise Internal Alignment (TC-LIA), a model-agnostic method that probes patch-token representations across the layers of a CLIP ViT-H/14 vision encoder.
  arXiv  Detail & Related papers  (2026-05-29T23:51:35Z)
• Beyond Pixels: Introspective and Interactive Grounding for Visualization Agents [3.1626173943755975]
  Vision-Language Models (VLMs) frequently misread values, hallucinate details, and confuse overlapping elements in charts.<n>Current approaches rely solely on pixel interpretation, creating a Pixel-Only Bottleneck.<n>Introspective and Interactive Visual Grounding (IVG) is a framework that combines spec-grounded introspection and view-grounded interaction.
  arXiv  Detail & Related papers  (2026-04-22T22:47:23Z)
• Nüwa: Mending the Spatial Integrity Torn by VLM Token Pruning [82.39668822222386]
  Vision token pruning has proven to be an effective acceleration technique for the efficient Vision Language Model (VLM)<n>We propose $textNwa$, a two-stage token pruning framework that enables efficient feature aggregation while maintaining spatial integrity.<n>Experiments demonstrate that $textNwa$ achieves SOTA performance on multiple VQA benchmarks (from 94% to 95%) and yields substantial improvements on visual grounding tasks (from 7% to 47%)
  arXiv  Detail & Related papers  (2026-02-03T00:51:03Z)
• CARES: Context-Aware Resolution Selector for VLMs [29.734101330721263]
  Large vision-language models (VLMs) commonly process images at native or high resolution to remain effective across tasks.<n>We introduce emphCARES, a lightweight preprocessing module that predicts the emphminimal sufficient input resolution.<n> CARES preserves task performance while reducing compute by up to 80%.
  arXiv  Detail & Related papers  (2025-10-22T11:44:31Z)
• Knowledge-based Visual Question Answer with Multimodal Processing, Retrieval and Filtering [55.49652734090316]
  A knowledge-based visual question answering (KB-VQA) requires visual language models (VLMs) to integrate visual understanding with external knowledge retrieval.<n>We propose a novel three-stage method, termed Wiki-PRF, including Processing, Retrieval and Filtering stages.<n> Experiments on benchmark datasets (E-VQA and InfoSeek) show significant improvements in answer quality, achieving state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-10-16T12:10:00Z)
• VisualOverload: Probing Visual Understanding of VLMs in Really Dense Scenes [36.370533774426555]
  We present VisualOverload, a visual question answering (VQA) benchmark comprising 2,720 question-answer pairs.<n>Unlike prior VQA datasets that typically focus on near global image understanding, VisualOverload challenges models to perform simple, knowledge-free vision tasks in densely populated scenes.<n>We observe that even the best model (o3) out of 37 tested models only achieves 19.6% accuracy on our hardest test split and overall 69.5% accuracy on all questions.
  arXiv  Detail & Related papers  (2025-09-29T18:00:25Z)
• WikiMixQA: A Multimodal Benchmark for Question Answering over Tables and Charts [14.966795545558474]
  This paper introduces WikiMixQA, a benchmark for evaluating cross-modal reasoning over tables and charts extracted from 4,000 Wikipedia pages.<n>We evaluate 12 state-of-the-art vision-language models, revealing that while proprietary models achieve 70% accuracy when provided with direct context, their performance deteriorates significantly when retrieval from long documents is required.
  arXiv  Detail & Related papers  (2025-06-18T16:09:18Z)
• DeepDiver: Adaptive Search Intensity Scaling via Open-Web Reinforcement Learning [73.68685269970844]
  We introduce WebPuzzle, a training and 275-sample test benchmark that evaluates information seeking on the live internet.<n>We develop DeepDiver, a reinforcement-learning framework that cultivates Search Intensity Scaling (SIS)-an emergent ability to escalate search frequency and depth.<n>We detail DeepDiver's curriculum from cold-start SFT to a well designed RL procedure, and show that its seeking policy generalized from closed-ended queries to open-ended generation such as long-form writing.
  arXiv  Detail & Related papers  (2025-05-30T08:15:39Z)
• DocVLM: Make Your VLM an Efficient Reader [16.11678886224833]
  We introduce DocVLM, a method that integrates an OCR-based modality into Vision-Language Models (VLMs)<n>Our approach employs an OCR encoder to capture textual content and layout, compressing these into a compact set of learned queries incorporated into the VLM.<n>In limited-token regimes (448$times$448), DocVLM with 64 learned queries improves DocVQA results from 56.4% to 86.6% when integrated with InternVL2 and from 84.4% to 91.2% with Qwen2-VL.
  arXiv  Detail & Related papers  (2024-12-11T19:35:06Z)
• Trust but Verify: Programmatic VLM Evaluation in the Wild [62.14071929143684]
  Programmatic VLM Evaluation (PROVE) is a new benchmarking paradigm for evaluating VLM responses to open-ended queries. We benchmark the helpfulness-truthfulness trade-offs of a range ofVLMs on PROVE, finding that very few are in-fact able to achieve a good balance between the two.
  arXiv  Detail & Related papers  (2024-10-17T01:19:18Z)
• ViTextVQA: A Large-Scale Visual Question Answering Dataset for Evaluating Vietnamese Text Comprehension in Images [1.2529442734851663]
  Visual Question Answerinng (VQA) is a complicated task that requires the capability of simultaneously processing natural language and images.<n>We introduce the first large-scale dataset in Vietnamese specializing in the ability to understand scene text.
  arXiv  Detail & Related papers  (2024-04-16T15:28:30Z)
• Look Before You Leap: A Universal Emergent Decomposition of Retrieval Tasks in Language Models [58.57279229066477]
  We study how language models (LMs) solve retrieval tasks in diverse situations. We introduce ORION, a collection of structured retrieval tasks spanning six domains. We find that LMs internally decompose retrieval tasks in a modular way.
  arXiv  Detail & Related papers  (2023-12-13T18:36:43Z)
• Rephrase, Augment, Reason: Visual Grounding of Questions for Vision-Language Models [59.05769810380928]
  Rephrase, Augment and Reason (RepARe) is a gradient-free framework that extracts salient details about the image using the underlying vision-language model. We show that RepARe can result in a 3.85% (absolute) increase in zero-shot accuracy on VQAv2, 6.41%, and 7.94% points increase on A-OKVQA, and VizWiz respectively.
  arXiv  Detail & Related papers  (2023-10-09T16:57:57Z)
• Dynamic Focus-aware Positional Queries for Semantic Segmentation [94.6834904076914]
  We propose a simple yet effective query design for semantic segmentation termed Dynamic Focus-aware Positional Queries. Our framework achieves SOTA performance and outperforms Mask2former by clear margins of 1.1%, 1.9%, and 1.1% single-scale mIoU with ResNet-50, Swin-T, and Swin-B backbones.
  arXiv  Detail & Related papers  (2022-04-04T05:16:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.