Reranking with Compressed Document Representation

• URL: http://arxiv.org/abs/2505.15394v1
• Date: Wed, 21 May 2025 11:35:11 GMT
• Title: Reranking with Compressed Document Representation
• Abstract summary: We reduce the input size by compressing documents into fixed-size embedding representations.<n>We then teach a reranker to use compressed inputs by distillation.<n>Although based on a billion-size model, our trained reranker using this compressed input can challenge smaller rerankers in terms of both effectiveness and efficiency.
• Score: 7.894770087349908
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Reranking, the process of refining the output of a first-stage retriever, is often considered computationally expensive, especially with Large Language Models. Borrowing from recent advances in document compression for RAG, we reduce the input size by compressing documents into fixed-size embedding representations. We then teach a reranker to use compressed inputs by distillation. Although based on a billion-size model, our trained reranker using this compressed input can challenge smaller rerankers in terms of both effectiveness and efficiency, especially for long documents. Given that text compressors are still in their early development stages, we view this approach as promising.

Related papers

• Compression Beyond the Uncompressed: A Two-Stage Training Recipe for Soft Context Compression in RAG [24.94881830262324]
  We propose DEX-Comp, a two-stage training recipe for RAG inference.<n>DEX-Comp compresses retrieved contexts by $16times$ and accelerates inference by $4times$--$24times$.<n>On five open-domain QA benchmarks at retrieval depths, DEX-Comp compresses retrieved contexts by $16times$ and accelerates inference by $4times$--$24times$.
  arXiv  Detail & Related papers  (2026-09-04T13:53:43Z)
• RAGOCR: Optical Compression of Retrieval-Augmented Text via Visual Representation [7.540266838459537]
  RAGOCR is a novel framework that compresses retrieved documents into compact visual representations conditioned on the input query.<n>We show that RAGOCR surpasses naive RAG by over 15% in accuracy while requiring only one-eighth the number of input tokens.
  arXiv  Detail & Related papers  (2026-08-01T16:59:49Z)
• Layer-wise Token Compression for Efficient Document Reranking [18.48737466474846]
  Cross-encoder rerankers suffer from high computational costs due to processing long query-document sequences at inference time.<n>We propose Layer-wise Token Compression, which applies adaptive token pooling at intermediate transformer layers.<n>We show that compression at middle layers preserves ranking quality while increasing inference QPS by up to 25% for passage ranking and up to 116% for document ranking.
  arXiv  Detail & Related papers  (2026-05-20T03:52:31Z)
• Efficient Listwise Reranking with Compressed Document Representations [10.623994225850046]
  We introduce RRK, an efficient listwise reranker compressing documents into multi-token fixed-size embedding representations.<n>Our simple training via distillation shows that this combination of rich compressed representations and listwise reranking yields a highly efficient and effective system.
  arXiv  Detail & Related papers  (2026-04-29T09:48:44Z)
• Arbitrary Ratio Feature Compression via Next Token Prediction [52.10426317889982]
  Arbitrary Ratio Feature Compression (ARFC) framework supports any compression ratio with a single model.<n>ARC is an auto-regressive model that performs compression via next-gressive prediction.<n>MoS module refines the compressed tokens by utilizing multiple compression results.<n>ERGC is integrated into the training process to preserve semantic and structural relationships during compression.
  arXiv  Detail & Related papers  (2026-02-12T02:38:57Z)
• Proxy Compression for Language Modeling [58.904023114033954]
  proxy compression is an alternative training scheme that preserves the efficiency benefits of compressed inputs.<n>Experiments on code language modeling demonstrate that proxy compression substantially improves training efficiency.<n>As model scale increases, proxy-trained models eventually match or rival tokenizer approaches.
  arXiv  Detail & Related papers  (2026-02-04T07:36:46Z)
• Noise-Robust Abstractive Compression in Retrieval-Augmented Language Models [0.0]
  Abstractive compression utilizes smaller langauge models to condense query-relevant context.<n> retrieved documents often include information that is either irrelevant to answering the query or misleading due to factual incorrect content.<n>ACoRN excels on datasets with many accuracy reducing documents, making it highly useful in real-world scenarios.
  arXiv  Detail & Related papers  (2025-11-19T00:51:27Z)
• ACoRN: Noise-Robust Abstractive Compression in Retrieval-Augmented Language Models [26.585985828583304]
  Abstractive compression utilizes smaller langauge models to condense query-relevant context.<n>Retrieved documents often include information that is either irrelevant to answering the query or misleading due to factual incorrect content.<n>This behavior indicates that abstractive compressors are more likely to omit important information essential for the correct answer.
  arXiv  Detail & Related papers  (2025-04-17T06:05:35Z)
• CODEPROMPTZIP: Code-specific Prompt Compression for Retrieval-Augmented Generation in Coding Tasks with LMs [6.936336826531964]
  Retrieval-Augmented Generation (RAG) enhances coding tasks by incorporating retrieved code examples into prompts.<n>Existing prompt compression techniques focus on natural language, lacking tailored solutions for code.<n>We propose CodePromptZip, a framework that compresses code examples before integrating into RAG.
  arXiv  Detail & Related papers  (2025-02-19T23:15:23Z)
• Efficient Long Context Language Model Retrieval with Compression [57.09163579304332]
  Long Context Language Models (LCLMs) have emerged as a new paradigm to perform Information Retrieval (IR)<n>We propose a new compression approach tailored for LCLM retrieval, which is trained to maximize the retrieval performance while minimizing the length of the compressed passages.<n>We show that CoLoR improves the retrieval performance by 6% while compressing the in-context size by a factor of 1.91.
  arXiv  Detail & Related papers  (2024-12-24T07:30:55Z)
• In-Context Former: Lightning-fast Compressing Context for Large Language Model [48.831304302467004]
  In this paper, we propose a new approach to compress the long input contexts of Transformer-based large language models (LLMs) We use the cross-attention mechanism and a small number of learnable digest tokens to condense information from the contextual word embeddings. Experimental results indicate that our method requires only 1/32 of the floating-point operations of the baseline during compression and improves processing speed by 68 to 112 times.
  arXiv  Detail & Related papers  (2024-06-19T15:14:55Z)
• What Operations can be Performed Directly on Compressed Arrays, and with What Error? [1.3307486544794784]
  We develop a lossy compressor that allows a dozen fairly fundamental operations directly on compressed data. We evaluate it on three non-trivial applications, choosing different number systems for internal representation.
  arXiv  Detail & Related papers  (2024-06-17T05:01:09Z)
• Context Compression for Auto-regressive Transformers with Sentinel Tokens [37.07722536907739]
  We propose a plug-and-play approach that is able to incrementally compress the intermediate activation of a specified span of tokens into compact ones. Experiments on both in-domain language modeling and zero-shot open-ended document generation demonstrate the advantage of our approach.
  arXiv  Detail & Related papers  (2023-10-12T09:18:19Z)
• RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentation [61.53695868960846]
  We propose compressing retrieved documents into textual summaries prior to in-context integration. This not only reduces the computational costs but also relieves the burden of LMs to identify relevant information in long retrieved documents. We show that our compressors trained for one LM can transfer to other LMs on the language modeling task and provide summaries largely faithful to the retrieved documents.
  arXiv  Detail & Related papers  (2023-10-06T17:55:36Z)
• Quick Dense Retrievers Consume KALE: Post Training Kullback Leibler Alignment of Embeddings for Asymmetrical dual encoders [89.29256833403169]
  We introduce Kullback Leibler Alignment of Embeddings (KALE), an efficient and accurate method for increasing the inference efficiency of dense retrieval methods. KALE extends traditional Knowledge Distillation after bi-encoder training, allowing for effective query encoder compression without full retraining or index generation. Using KALE and asymmetric training, we can generate models which exceed the performance of DistilBERT despite having 3x faster inference.
  arXiv  Detail & Related papers  (2023-03-31T15:44:13Z)
• Unrolled Compressed Blind-Deconvolution [77.88847247301682]
  sparse multichannel blind deconvolution (S-MBD) arises frequently in many engineering applications such as radar/sonar/ultrasound imaging. We propose a compression method that enables blind recovery from much fewer measurements with respect to the full received signal in time.
  arXiv  Detail & Related papers  (2022-09-28T15:16:58Z)
• Implicit Neural Representations for Image Compression [103.78615661013623]
  Implicit Neural Representations (INRs) have gained attention as a novel and effective representation for various data types. We propose the first comprehensive compression pipeline based on INRs including quantization, quantization-aware retraining and entropy coding. We find that our approach to source compression with INRs vastly outperforms similar prior work.
  arXiv  Detail & Related papers  (2021-12-08T13:02:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.