Survival-Guided Length Control for Efficient Diffusion Language Models

• URL: http://arxiv.org/abs/2608.26374v1
• Date: Wed, 26 Aug 2026 19:59:42 GMT
• Title: Survival-Guided Length Control for Efficient Diffusion Language Models
• Abstract summary: We recast length selection as a discrete-time survival problem over the end-of-sequence token.<n>We propose a plug-in, training-free length predictor that can be added to any existing DLM.
• Score: 15.49436655340166
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion language models (DLMs) generate text by iteratively denoising masked sequences, but standard decoding either fixes the sequence length or relies on ad hoc stopping rules, often leading to unnecessary denoising steps. We recast length selection as a discrete-time survival problem over the end-of-sequence token and propose a plug-in, training-free length predictor that can be added to any existing DLM. Across reasoning and code-generation benchmarks, survival-guided length decoding speeds up inference by up to 7 times while preserving task accuracy. We further find that predicted lengths vary widely even within the same dataset, making model performance sensitive to the chosen length.

Related papers

• Predict, Don't Iterate: Efficient Adaptive-Length Infilling for Diffusion Language Models [78.02669593911784]
  Diffusion language models (DLMs) have emerged as a promising alternative to the auto-regressive paradigm.<n>We propose PILL (Probing-based InfiLling with preset-Length-free decoding), an efficient infilling method for DLMs.
  arXiv  Detail & Related papers  (2026-09-02T04:52:01Z)
• Beyond Tokenization: Direct Timestep Embedding and Contrastive Alignment for Time-Series Question Answering [13.074536659496362]
  Recent advances in large language models (LLMs) have given rise to time-series question answering (TSQA)<n>TSQA formulates time-series analysis as natural-language question answering.<n>To address this challenge, we propose CADE, a novel framework for TSQA built upon two key components: direct timestep embedding and semantic alignment.
  arXiv  Detail & Related papers  (2026-06-17T12:07:23Z)
• Diffusion Large Language Models for Visual Speech Recognition [56.81307584718608]
  We propose the first Diffusion Large Language Model (DLLM)-based Visual Speech Recognition (VSR) framework.<n>DLLM-VSR commits high-confidence positions early and uses the committed tokens as bidirectional context to refine ambiguous ones.<n>We develop length-guided candidate decoding, which uses video duration to construct plausible transcript-length hypotheses.<n>The proposed method achieves a state-of-the-art WER of 19.5% on LRS3 using only its labeled training data.
  arXiv  Detail & Related papers  (2026-05-27T13:22:08Z)
• TS-Haystack: A Multi-Scale Retrieval Benchmark for Time Series Language Models [4.387988928531881]
  Time Series Language Models (TSLMs) are emerging as unified models for reasoning over continuous signals in natural language.<n>Existing models are typically trained and evaluated on short sequences, while real-world time-series sensor streams can span millions of datapoints.<n>We introduce TS-Haystack, a long-context temporal retrieval benchmark comprising ten task types across four categories.
  arXiv  Detail & Related papers  (2026-02-15T15:50:02Z)
• Length-Adaptive Interest Network for Balancing Long and Short Sequence Modeling in CTR Prediction [50.094751096858204]
  LAIN is a plug-and-play framework that incorporates sequence length as a conditioning signal to balance long- and short-sequence modeling.<n>Our work offers a general, efficient, and deployable solution to mitigate length-induced bias in sequential recommendation.
  arXiv  Detail & Related papers  (2026-01-27T03:14:20Z)
• Can LLMs Track Their Output Length? A Dynamic Feedback Mechanism for Precise Length Regulation [50.821215666749545]
  Large Language Models (LLMs) often fail to accurately measure their response lengths, leading to poor adherence to length constraints.<n>We propose a novel length regulation approach that incorporates dynamic length feedback during generation.<n> Experiments on summarization and biography tasks show our training-free approach significantly improves precision in achieving target token, word, or sentence counts.
  arXiv  Detail & Related papers  (2026-01-05T03:49:14Z)
• DiffuSpec: Unlocking Diffusion Language Models for Speculative Decoding [66.40658898418316]
  We present DiffuSpec, a training-free drop-in framework that uses a pretrained diffusion language model (DLM) to produce multi-token drafts in a single forward pass.<n>Across benchmarks, DiffuSpec yields up to 3x wall-clock speedup, establishing diffusion-based drafting as a robust alternative to autoregressive drafters for speculative decoding.
  arXiv  Detail & Related papers  (2025-09-28T07:00:15Z)
• Beyond Fixed: Training-Free Variable-Length Denoising for Diffusion Large Language Models [74.15250326312179]
  Diffusion Large Language Models offer efficient parallel generation and capable global modeling.<n>The dominant application ofDLLMs is hindered by the need for a statically predefined generation length.<n>We introduce DAEDAL, a novel training-free denoising strategy that enables Dynamic Adaptive Length Expansion.
  arXiv  Detail & Related papers  (2025-08-01T17:56:07Z)
• Hansel: Output Length Controlling Framework for Large Language Models [3.053720182019654]
  Hansel is an efficient framework for length control in large language models.<n>The framework can be applied to any pre-trained LLMs during the finetuning stage of the model.
  arXiv  Detail & Related papers  (2024-12-18T16:52:38Z)
• Precise Length Control in Large Language Models [1.3654846342364308]
  Large Language Models (LLMs) are increasingly used in production systems.<n>We propose a method to adapt pre-trained decoder-only LLMs for precise control of response length.
  arXiv  Detail & Related papers  (2024-12-16T16:22:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.