Parallel Context-of-Experts Decoding for Retrieval Augmented Generation

• URL: http://arxiv.org/abs/2601.08670v1
• Date: Tue, 13 Jan 2026 15:46:59 GMT
• Title: Parallel Context-of-Experts Decoding for Retrieval Augmented Generation
• Abstract summary: We propose a training-free framework that shifts evidence aggregation from the attention mechanism to the decoding.<n>We propose Parallel Context-of-Experts Decoding (Pced), a training-free framework that shifts evidence aggregation from the attention mechanism to the decoding.
• Score: 11.887068025002362
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Retrieval Augmented Generation faces a trade-off: concatenating documents in a long prompt enables multi-document reasoning but creates prefill bottlenecks, while encoding document KV caches separately offers speed but breaks cross-document interaction. We propose Parallel Context-of-Experts Decoding (Pced), a training-free framework that shifts evidence aggregation from the attention mechanism to the decoding. Pced treats retrieved documents as isolated "experts", synchronizing their predictions via a novel retrieval-aware contrastive decoding rule that weighs expert logits against the model prior. This approach recovers cross-document reasoning capabilities without constructing a shared attention across documents.

Related papers

• MARDoc: A Memory-Aware Refinement Agent Framework for Multimodal Long Document QA [23.25080415779712]
  We propose a Memory-Aware Refinement Agent framework that decouples long-document QA into three specialized agents.<n>Across iterations, the agents rely on a dynamically updated structured memory rather than a full accumulated interaction history.<n>Experiments on MMLongBench-Doc and DocBench show that MARDoc achieves strong results, outperforming same-backbone baselines.
  arXiv  Detail & Related papers  (2026-06-04T06:23:01Z)
• ICICLE: Expanding Retrieval with In-Context Documents [28.99813139934325]
  Generative retrieval (GR) maps queries directly to document identifiers (docids) using parametric knowledge.<n>We propose ICICLE, an in-context indexing framework that performs source-aware docid generation over both parametric memory and context-provided document-docid pairs.
  arXiv  Detail & Related papers  (2026-05-26T12:01:44Z)
• DocSync: Agentic Documentation Maintenance via Critic-Guided Reflexion [0.0]
  DocSync is an agentic workflow that frames documentation maintenance as a structurally grounded, iterative generation task.<n>We empirically evaluate a resource-constrained implementation of DocSync-using a LoRA-adapted small language model.<n>Our findings demonstrate that this AST-aware agentic approach substantially outperforms standard encoder-decoder baselines.
  arXiv  Detail & Related papers  (2026-05-04T02:41:33Z)
• Training-Free Acceleration for Document Parsing Vision-Language Model with Hierarchical Speculative Decoding [102.88996030431662]
  We propose a training-free and highly efficient acceleration method for document parsing tasks.<n>Inspired by speculative decoding, we employ a lightweight document parsing pipeline as a draft model to predict batches of future tokens.<n>We demonstrate the effectiveness of our approach on the general-purpose OmniDocBench.
  arXiv  Detail & Related papers  (2026-02-13T14:22:10Z)
• Codetations: Intelligent, Persistent Notes and UIs for Programs and Other Documents [0.85830154886823]
  We present Codetations, a system that helps developers contextualize documents with rich notes and tools.<n>Unlike previous approaches, notes in Codetations stay outside the document to prevent code clutter, attaching to spans in the document using a hybrid edit-tracking/LLM-based method.<n>Their content is dynamic, interactive, and synchronized with code changes.
  arXiv  Detail & Related papers  (2025-04-25T21:33:25Z)
• Learning Refined Document Representations for Dense Retrieval via Deliberate Thinking [58.69615583599489]
  Deliberate Thinking based Retriever (Debater) is a novel approach that enhances document representations by incorporating a step-by-step thinking process.<n>Debater significantly outperforms existing methods across several retrieval benchmarks.
  arXiv  Detail & Related papers  (2025-02-18T15:56:34Z)
• Contextual Document Embeddings [77.22328616983417]
  We propose two complementary methods for contextualized document embeddings. First, an alternative contrastive learning objective that explicitly incorporates the document neighbors into the intra-batch contextual loss. Second, a new contextual architecture that explicitly encodes neighbor document information into the encoded representation.
  arXiv  Detail & Related papers  (2024-10-03T14:33:34Z)
• CREPE: Coordinate-Aware End-to-End Document Parser [13.530212337717515]
  We formulate an OCR-free sequence generation model for visual document understanding (VDU) Our model not only parses text from document images but also extracts the spatial coordinates of the text based on the multi-head architecture. Named as Coordinate-aware End-to-end Document. CREPE, our method uniquely integrates these capabilities by introducing a special token for OCR text.
  arXiv  Detail & Related papers  (2024-05-01T00:30:13Z)
• Natural Logic-guided Autoregressive Multi-hop Document Retrieval for Fact Verification [21.04611844009438]
  We propose a novel retrieve-and-rerank method for multi-hop retrieval. It consists of a retriever that jointly scores documents in the knowledge source and sentences from previously retrieved documents. It is guided by a proof system that dynamically terminates the retrieval process if the evidence is deemed sufficient.
  arXiv  Detail & Related papers  (2022-12-10T11:32:38Z)
• Learning Diverse Document Representations with Deep Query Interactions for Dense Retrieval [79.37614949970013]
  We propose a new dense retrieval model which learns diverse document representations with deep query interactions. Our model encodes each document with a set of generated pseudo-queries to get query-informed, multi-view document representations.
  arXiv  Detail & Related papers  (2022-08-08T16:00:55Z)
• Multilevel Text Alignment with Cross-Document Attention [59.76351805607481]
  Existing alignment methods operate at a single, predefined level. We propose a new learning approach that equips previously established hierarchical attention encoders for representing documents with a cross-document attention component.
  arXiv  Detail & Related papers  (2020-10-03T02:52:28Z)
• Fast(er) Reconstruction of Shredded Text Documents via Self-Supervised Deep Asymmetric Metric Learning [62.34197797857823]
  A central problem in automatic reconstruction of shredded documents is the pairwise compatibility evaluation of the shreds. This work proposes a scalable deep learning approach for measuring pairwise compatibility in which the number of inferences scales linearly. Our method has accuracy comparable to the state-of-the-art with a speed-up of about 22 times for a test instance with 505 shreds.
  arXiv  Detail & Related papers  (2020-03-23T03:22:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.