Reflection-Window Decoding: Text Generation with Selective Refinement

• URL: http://arxiv.org/abs/2502.03678v2
• Date: Mon, 10 Mar 2025 19:34:32 GMT
• Title: Reflection-Window Decoding: Text Generation with Selective Refinement
• Authors: Zeyu Tang, Zhenhao Chen, Loka Li, Xiangchen Song, Yunlong Deng, Yifan Shen, Guangyi Chen, Peter Spirtes, Kun Zhang,
• Abstract summary: The autoregressive decoding for text generation in large language models (LLMs) is inherently suboptimal due to the lack of a built-in mechanism to perform refinement and/or correction of the generated content.<n>We propose an approach that incorporates a sliding reflection window and a pausing criterion, such that refinement and generation can be carried out interchangeably as the decoding proceeds.
• Score: 14.943512664474143
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The autoregressive decoding for text generation in large language models (LLMs), while widely used, is inherently suboptimal due to the lack of a built-in mechanism to perform refinement and/or correction of the generated content. In this paper, we consider optimality in terms of the joint probability over the generated response, when jointly considering all tokens at the same time. We theoretically characterize the potential deviation of the autoregressively generated response from its globally optimal counterpart that is of the same length. Our analysis suggests that we need to be cautious when noticeable uncertainty arises during text generation, which may signal the sub-optimality of the generation history. To address the pitfall of autoregressive decoding for text generation, we propose an approach that incorporates a sliding reflection window and a pausing criterion, such that refinement and generation can be carried out interchangeably as the decoding proceeds. Our selective refinement framework strikes a balance between efficiency and optimality, and our extensive experimental results demonstrate the effectiveness of our approach.

Related papers

• Towards Better Code Generation: Adaptive Decoding with Uncertainty Guidance [28.99265405319943]
  We introduce AdaDec, an adaptive decoding framework guided by token-level uncertainty quantified via Shannon entropy.<n>AdaDec achieves up to a 15.5% improvement in Pass@1 accuracy compared to greedy decoding, matches or outperforms traditional beam search.
  arXiv  Detail & Related papers  (2025-06-10T16:49:46Z)
• Constrained Auto-Regressive Decoding Constrains Generative Retrieval [71.71161220261655]
  Generative retrieval seeks to replace traditional search index data structures with a single large-scale neural network. In this paper, we examine the inherent limitations of constrained auto-regressive generation from two essential perspectives: constraints and beam search.
  arXiv  Detail & Related papers  (2025-04-14T06:54:49Z)
• Synthetic Text Generation for Training Large Language Models via Gradient Matching [27.74603049449281]
  We propose the first theoretically rigorous approach for generating synthetic human-readable text. In doing so, the generated synthetic text can guarantee convergence of the model to a close neighborhood of the solution obtained by fine-tuning on real data.
  arXiv  Detail & Related papers  (2025-02-24T19:49:15Z)
• Adaptive Contrastive Search: Uncertainty-Guided Decoding for Open-Ended Text Generation [0.20971479389679337]
  We introduce adaptive contrastive search, a novel decoding strategy extending contrastive search. Our findings indicate performance enhancement in both aspects, across different model architectures and datasets.
  arXiv  Detail & Related papers  (2024-07-26T12:23:54Z)
• Positive Text Reframing under Multi-strategy Optimization [2.6345343328000856]
  We propose a framework to generate fluent, diverse and task-constrained reframing text.<n>Our framework achieves significant improvements on unconstrained and controlled positive reframing tasks.
  arXiv  Detail & Related papers  (2024-07-25T10:58:42Z)
• Improving Open-Ended Text Generation via Adaptive Decoding [6.746656673563492]
  This study introduces adaptive decoding, a mechanism that dynamically empowers language models to ascertain a sensible candidate set during generation. Experimental results reveal that our method balances diversity and coherence well. Our method can potentially improve the reasoning ability of language models.
  arXiv  Detail & Related papers  (2024-02-28T10:38:21Z)
• Lightweight reranking for language model generations [26.942659041383596]
  We present a novel approach for reranking Large Language Models (LLMs) generations. Unlike other techniques that might involve additional inferences or training a specialized reranker, our approach relies on easy to compute pairwise statistics. We show strong improvements for selecting the best k generations for code generation tasks as well as robust improvements for the best generation for the tasks of autoformalization, summarization, and translation.
  arXiv  Detail & Related papers  (2023-07-11T17:51:48Z)
• PLANNER: Generating Diversified Paragraph via Latent Language Diffusion Model [37.2192243883707]
  We propose PLANNER, a model that combines latent semantic diffusion with autoregressive generation to generate fluent text. Results on semantic generation, text completion and summarization show its effectiveness in generating high-quality long-form text.
  arXiv  Detail & Related papers  (2023-06-05T01:36:39Z)
• EEL: Efficiently Encoding Lattices for Reranking [44.77383151122229]
  We use Transformers to efficiently encode lattices of generated outputs. We combine this approach with a new class of token-factored rerankers (TFRs) Our results show both substantial speedup compared to naive reranking and often better performance on downstream metrics than comparable approaches.
  arXiv  Detail & Related papers  (2023-06-01T17:45:32Z)
• Look-back Decoding for Open-Ended Text Generation [62.53302138266465]
  We propose Look-back, an improved decoding algorithm that tracks the distribution distance between current and historical decoding steps. Look-back can automatically predict potential repetitive phrase and topic drift, and remove tokens that may cause the failure modes. We perform decoding experiments on document continuation and story generation, and demonstrate that Look-back is able to generate more fluent and coherent text.
  arXiv  Detail & Related papers  (2023-05-22T20:42:37Z)
• Towards Robust Speech-to-Text Adversarial Attack [78.5097679815944]
  This paper introduces a novel adversarial algorithm for attacking the state-of-the-art speech-to-text systems, namely DeepSpeech, Kaldi, and Lingvo. Our approach is based on developing an extension for the conventional distortion condition of the adversarial optimization formulation. Minimizing over this metric, which measures the discrepancies between original and adversarial samples' distributions, contributes to crafting signals very close to the subspace of legitimate speech recordings.
  arXiv  Detail & Related papers  (2021-03-15T01:51:41Z)
• Improving Adversarial Text Generation by Modeling the Distant Future [155.83051741029732]
  We consider a text planning scheme and present a model-based imitation-learning approach to alleviate the aforementioned issues. We propose a novel guider network to focus on the generative process over a longer horizon, which can assist next-word prediction and provide intermediate rewards for generator optimization.
  arXiv  Detail & Related papers  (2020-05-04T05:45:13Z)
• Improve Variational Autoencoder for Text Generationwith Discrete Latent Bottleneck [52.08901549360262]
  Variational autoencoders (VAEs) are essential tools in end-to-end representation learning. VAEs tend to ignore latent variables with a strong auto-regressive decoder. We propose a principled approach to enforce an implicit latent feature matching in a more compact latent space.
  arXiv  Detail & Related papers  (2020-04-22T14:41:37Z)
• Heavy-tailed Representations, Text Polarity Classification & Data Augmentation [11.624944730002298]
  We develop a novel method to learn a heavy-tailed embedding with desirable regularity properties. A classifier dedicated to the tails of the proposed embedding is obtained which performance outperforms the baseline. Numerical experiments on synthetic and real text data demonstrate the relevance of the proposed framework.
  arXiv  Detail & Related papers  (2020-03-25T19:24:05Z)
• Self-Adversarial Learning with Comparative Discrimination for Text Generation [111.18614166615968]
  We propose a novel self-adversarial learning (SAL) paradigm for improving GANs' performance in text generation. During training, SAL rewards the generator when its currently generated sentence is found to be better than its previously generated samples. Experiments on text generation benchmark datasets show that our proposed approach substantially improves both the quality and the diversity.
  arXiv  Detail & Related papers  (2020-01-31T07:50:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.