Lost in Decoding? Reproducing and Stress-Testing the Look-Ahead Prior in Generative Retrieval

• URL: http://arxiv.org/abs/2604.23396v1
• Date: Sat, 25 Apr 2026 17:58:15 GMT
• Title: Lost in Decoding? Reproducing and Stress-Testing the Look-Ahead Prior in Generative Retrieval
• Abstract summary: Generative retrieval (GR) ranks documents by autoregressively generating document identifiers.<n>Many GR methods rely on trie-latency beam search, they are vulnerable to early pruning of relevant prefixes under finite-beam decoding.<n>Planning Ahead in Generative Retrieval (PAG) mitigates this failure mode by using simultaneous decoding to compute a document-level look-ahead prior that guides subsequent sequential decoding.
• Score: 48.348257016776195
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generative retrieval (GR) ranks documents by autoregressively generating document identifiers. Because many GR methods rely on trie-constrained beam search, they are vulnerable to early pruning of relevant prefixes under finite-beam decoding. Planning Ahead in Generative Retrieval (PAG) mitigates this failure mode by using simultaneous decoding to compute a document-level look-ahead prior that guides subsequent sequential decoding. We reproduce PAG at inference time and stress-test its decoding behavior. Using the authors' released checkpoint and identifier/trie artifacts under the reported decoding setup, we reproduce the main effectiveness results on MS MARCO Dev and TREC-DL 2019/2020, and corroborate the reported beam-size-latency trade-off in our hardware setting. Beyond reproduction, we introduce plan drift diagnostics that quantify how intent-preserving query variations alter the planner's top-n candidate set and highest-weight planner tokens, and how these changes affect guided decoding. We find that PAG's planning signal is brittle under lexical surface-form variation: intent-preserving typos can trigger plan collapse, where the planned candidate pool shifts enough that the look-ahead bonus provides little useful guidance, effectively reverting decoding toward weaker unguided search. We further evaluate fixed-index cross-lingual robustness using non-English mMARCO queries against an English index, and assess query-side mitigation strategies that require no re-indexing; query translation provides the strongest recovery in our setting. Overall, our results confirm PAG's reported effectiveness and the benefit of planning-guided decoding under the released inference setup, while showing that these gains depend on the stability of the planning signal under realistic query variation and query-document mismatch.

Related papers

• WIDE: Wildcard Inference with Dynamic Expansion for Cross-Modal Generative Retrieval [11.681276883454288]
  We propose Wildcard Inference with Dynamic Expansion (WIDE) to overcome the problem of forced hallucination in cross-modal retrieval.<n>During the decoding generation phase, Asymmetry-aware Wildcard Decoding (AWD) detects semantic blind spots and emits wildcards instead of forced deterministic identifiers.<n>Blind-Spot Re-ranking (BSR) evaluates the expanded candidate pool using a hybrid scoring mechanism that combines discrete generation confidence with continuous semantic similarity.
  arXiv  Detail & Related papers  (2026-09-03T08:55:05Z)
• GUIDE: Generative Unsupervised Chinese Query Correction via Phonetic and Visual Shared-ID Encoding [14.765897981494186]
  Chinese query correction (CQC) is important for search and query recommendation on content platforms.<n>We propose textscGUIDE, a generative unsupervised framework for CQC based on a confuse-then-clarify paradigm.<n>textscGUIDE encodes phonetically or visually confusable characters with shared-IDs and reconstructs the original query with an encoder-decoder architecture.
  arXiv  Detail & Related papers  (2026-08-26T03:54:26Z)
• Beyond Fixed Depths and Widths: Optimizing Textual Decoding Tries in LLM-based Generative Recommendation [59.20704286413859]
  We propose a novel framework that co-designs textual term IDs and their underlying decoding trie.<n> BONSAI achieves up to a 21.6% relative improvement over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-07-18T04:31:02Z)
• Deeper is Not Always Better: Mitigating the Alignment Tax via Confident Layer Decoding [92.1521161575198]
  We introduce Confident Decoding, a training-free decoding strategy that dynamically selects the most reliable near-final layer.<n> Experiments across dense and Mixture-of-Experts LLMs demonstrate consistent gains on challenging reasoning benchmarks.
  arXiv  Detail & Related papers  (2026-06-20T07:03:26Z)
• Closing the Indexing-Decoding Gap in Multimodal Generative Retrieval via Prefix Retention Optimization [68.48718919047127]
  Multimodal generative retrieval formulates multimodal retrieval as discrete identifier generation, eliminating the need for explicit similarity search over external embeddings.<n>Existing approaches construct identifiers via residual quantization and decode them with trie-constrained beam search.<n>This combination introduces an indexing-decoding gap: identifier learning objectives, including reconstruction and contrastive losses, do not explicitly enforce prefix discriminability during decoding.<n>We propose PRO, a unified framework comprising three mechanisms: (i) prefix ranking distillation aligns quantized prefixes with those induced by pre-quantization embeddings using a listwise loss; (ii) vocabulary scheduling increases codebook
  arXiv  Detail & Related papers  (2026-06-08T09:15:47Z)
• A Parametric Memory Head for Continual Generative Retrieval [52.66674234249913]
  Generative information retrieval (GenIR) consolidates retrieval into a single neural model that decodes document identifiers (docids) directly from queries.<n>We show that sequential adaptation improves retrieval on newly added documents but substantially degrades performance on earlier slices.<n>We propose post-adaptation memory tuning (PAMT), a memory-only stabilization stage that augments an adapted model with a modular parametric memory head.
  arXiv  Detail & Related papers  (2026-04-25T17:38:51Z)
• Differentiable Semantic ID for Generative Recommendation [65.83703273297492]
  Generative recommendation provides a novel paradigm in which each item is represented by a discrete semantic ID (SID) learned from rich content.<n>In practice, SIDs are typically optimized only for content reconstruction rather than recommendation accuracy.<n>A natural approach is to make semantic indexing differentiable so that recommendation gradients can directly influence SID learning.<n>We propose DIGER, a first step toward effective differentiable semantic IDs for generative recommendation.
  arXiv  Detail & Related papers  (2026-01-27T15:34:11Z)
• Do LLMs Know They Are Being Tested? Evaluation Awareness and Incentive-Sensitive Failures in GPT-OSS-20B [1.948261185683419]
  We investigate whether "evaluation scent" inflates measured performance without commensurate capability gains.<n>We run six paired A/B scenarios that hold task content and decoding fixed while varying framing.<n>We provide a reproducible A/B framework (prompt banks, validators, per-run scores, scripts) and practical guidance.
  arXiv  Detail & Related papers  (2025-10-08T09:49:05Z)
• Reinforcement Speculative Decoding for Fast Ranking [9.584558586988953]
  Large Language Models (LLMs) have been widely adopted in ranking systems such as information retrieval (IR) systems and recommender systems (RSs)<n>We propose a Reinforcementive Decoding method for fast ranking inference of LLMs.
  arXiv  Detail & Related papers  (2025-05-23T02:25:26Z)
• Test-Time Alignment for Large Language Models via Textual Model Predictive Control [63.508812485566374]
  Textual Model Predictive Control (TMPC) is a novel predictive planning framework adapted for aligning Large Language Models at inference time.<n>TMPC is evaluated on three tasks with distinct segmentation properties: discourse-level translation, long-form response generation, and program synthesis.<n>Results demonstrate that TMPC consistently improves performance, highlighting the generality.
  arXiv  Detail & Related papers  (2025-02-28T07:24:33Z)
• Corrective Retrieval Augmented Generation [36.04062963574603]
  Retrieval-augmented generation (RAG) relies heavily on relevance of retrieved documents, raising concerns about how the model behaves if retrieval goes wrong. We propose the Corrective Retrieval Augmented Generation (CRAG) to improve the robustness of generation. CRAG is plug-and-play and can be seamlessly coupled with various RAG-based approaches.
  arXiv  Detail & Related papers  (2024-01-29T04:36:39Z)
• Noise-Robust Dense Retrieval via Contrastive Alignment Post Training [89.29256833403167]
  Contrastive Alignment POst Training (CAPOT) is a highly efficient finetuning method that improves model robustness without requiring index regeneration. CAPOT enables robust retrieval by freezing the document encoder while the query encoder learns to align noisy queries with their unaltered root. We evaluate CAPOT noisy variants of MSMARCO, Natural Questions, and Trivia QA passage retrieval, finding CAPOT has a similar impact as data augmentation with none of its overhead.
  arXiv  Detail & Related papers  (2023-04-06T22:16:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.