Related papers: Lost in a Single Vector: Improving Long-Document Retrieval with Chunk Evidence Aggregation

Lost in a Single Vector: Improving Long-Document Retrieval with Chunk Evidence Aggregation

URL: http://arxiv.org/abs/2606.18781v1
Date: Wed, 17 Jun 2026 07:44:04 GMT
Title: Lost in a Single Vector: Improving Long-Document Retrieval with Chunk Evidence Aggregation
Abstract summary: DICE is a training-free document-side strategy that splits documents into chunks, encodes them independently with a frozen model, and aggregates them back into a single vector.<n>On LongEmbed, DICE improves retrieval across four backbones, with the largest gains on slices beyond 4k tokens.<n>Across 12,779 filtered samples, DICE yields lower EDI than the single-vector baseline in 92.8% of cases.
Score: 59.438696079345426
License: http://creativecommons.org/licenses/by/4.0/
Abstract: Dense retrieval ranks one query vector against one document vector. On long documents, this interface can fail when a short but decisive span is weakened during document encoding before ranking. We study this failure mode as document-side early compression and introduce the Evidence Dilution Index (EDI) to measure how far a document-level representation falls below the strongest chunk-level evidence within the same gold document. Guided by this view, we propose DICE (Document Inference via Chunk Evidence), a training-free document-side strategy that splits documents into chunks, encodes them independently with a frozen model, and aggregates them back into a single vector while preserving the standard one-query-one-document interface. On LongEmbed, DICE improves retrieval across four backbones, with the largest gains on slices beyond 4k tokens: for Dream, Passkey >4k rises from 30.0 to 90.0 and Needle >4k from 23.3 to 74.0. Across 12,779 filtered samples, DICE yields lower EDI than the single-vector baseline in 92.8% of cases. These results establish document-level encoding as a practical and underexplored lever for long-document retrieval.

Related papers

REIGN: Refurbished Embeddings with Integrated Guidance Networks for Efficient Context-Length Scaling [0.0]
REIGN is a bi-encoder that operates on sequences of contextualised chunk embeddings from a frozen Guidance Network (GN) rather than on raw tokens.<n>We release a synthetic long-document retrieval benchmark for contrastive training and evaluation at long context lengths.<n>A paired significance test puts it on par with models 1.6-4.3x larger on the patent task, and it stays within 0.65 nDCG@10 of a 20x-larger model on LoCo.
arXiv Detail & Related papers (2026-08-30T16:50:38Z)
DocPC: Document-Level Visual Retrieval via Representative Page Composition [18.502834715003928]
We propose DocPC, a document-level visual retrieval framework based on Representative Page Composition.<n>We combine multi-positive contrastive learning with sparsely scheduled listwise optimization to handle multi-positive supervision prevalent at the document level.<n> DocPC-ColQwen achieves NDCG@5 of 44.09 on DocViRe, outperforming the strongest page-level baseline at 38.91 while reducing storage by 10.1x.
arXiv Detail & Related papers (2026-08-26T06:46:44Z)
DocAtlas: Long-Document Understanding as Mutable-State Interaction [67.90400271544678]
We present DocAtlas, a system that treats long-document understanding as a mutable-state information-seeking process.<n>Given a document and a question, DocAtlas exposes search, reading, note-taking, and review tools, maintains a hierarchical tree and note store, and updates both as the agent records evidence.<n>With GPT-5.4, DocAtlas reaches 71.4% on MMLongBench-Doc, exceeding the human-expert reference of 65.8%.
arXiv Detail & Related papers (2026-07-21T09:45:34Z)
CHaystack: Benchmarking Chinese Document Retrieval and VQA [0.998362415939221]
We introduce CHaystack, a Chinese DocumentVQA benchmark for large-scale document retrieval and question answering.<n>CHaystack covers four document categories, namely academic papers, advertisements, web pages, and real-world photographed documents.<n>We present CRAG, a Chinese DocumentVQA system that uses a VLM-based relevance filter to verify retrieved document images before answer generation.
arXiv Detail & Related papers (2026-05-30T06:59:33Z)
A Picture is Worth a Thousand Words? An Empirical Study of Aggregation Strategies for Visual Financial Document Retrieval [8.560410767537162]
Visual RAG treats documents as images and uses vision encoders to obtain vision patch tokens.<n>Hundreds of patch tokens per document create retrieval and storage challenges in a vector database.<n>This raises a critical question: does single-vector aggregation lose key information in financial documents?
arXiv Detail & Related papers (2026-05-14T08:53:40Z)
DeepRead: Document Structure-Aware Reasoning to Enhance Agentic Search [23.447631421934847]
DeepRead is a structure-aware document reasoning agent designed to operationalize document-native structural priors into actionable reasoning capabilities.<n>DeepRead elicits a human-like locate-then-read'' reasoning paradigm, effectively mitigating the context fragmentation inherent in traditional retrieval methods.
arXiv Detail & Related papers (2026-02-04T20:03:28Z)
Hierarchical Retrieval: The Geometry and a Pretrain-Finetune Recipe [42.35197658021889]
Dual encoder (DE) models, where a pair of matching query and document are embedded into similar vector representations, are widely used in information retrieval.<n>This paper investigates such limitations in the context of hierarchical retrieval (HR), where the document set has a hierarchical structure and the matching documents for a query are all of its ancestors.<n>We introduce a pretrain-finetune recipe that significantly improves long-distance retrieval without sacrificing performance on closer documents.
arXiv Detail & Related papers (2025-09-19T20:35:58Z)
Collapse of Dense Retrievers: Short, Early, and Literal Biases Outranking Factual Evidence [56.09494651178128]
Retrieval models are commonly used in Information Retrieval (IR) applications, such as Retrieval-Augmented Generation (RAG)<n>We quantify the impact of biases, such as a preference for shorter documents, on retrievers like Dragon+ and Contriever.<n>We uncover major vulnerabilities, showing retrievers favor shorter documents, early positions, repeated entities, and literal matches, all while ignoring the answer's presence!
arXiv Detail & Related papers (2025-03-06T23:23:13Z)
Learning Refined Document Representations for Dense Retrieval via Deliberate Thinking [58.69615583599489]
Deliberate Thinking based Retriever (Debater) is a novel approach that enhances document representations by incorporating a step-by-step thinking process.<n>Debater significantly outperforms existing methods across several retrieval benchmarks.
arXiv Detail & Related papers (2025-02-18T15:56:34Z)
Unifying Multimodal Retrieval via Document Screenshot Embedding [92.03571344075607]
Document Screenshot Embedding (DSE) is a novel retrieval paradigm that regards document screenshots as a unified input format.<n>We first craft the dataset of Wiki-SS, a 1.3M Wikipedia web page screenshots as the corpus to answer the questions from the Natural Questions dataset.<n>For example, DSE outperforms BM25 by 17 points in top-1 retrieval accuracy. Additionally, in a mixed-modality task of slide retrieval, DSE significantly outperforms OCR text retrieval methods by over 15 points in nDCG@10.
arXiv Detail & Related papers (2024-06-17T06:27:35Z)
DAPR: A Benchmark on Document-Aware Passage Retrieval [57.45793782107218]
We propose and name this task emphDocument-Aware Passage Retrieval (DAPR) While analyzing the errors of the State-of-The-Art (SoTA) passage retrievers, we find the major errors (53.5%) are due to missing document context. Our created benchmark enables future research on developing and comparing retrieval systems for the new task.
arXiv Detail & Related papers (2023-05-23T10:39:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.