The Path Matters: Learning a Token-Commitment Policy for Diffusion Language Models

• URL: http://arxiv.org/abs/2605.24697v1
• Date: Sat, 23 May 2026 18:23:46 GMT
• Title: The Path Matters: Learning a Token-Commitment Policy for Diffusion Language Models
• Abstract summary: We argue that token commitment can be learned as a reusable trace-state policy.<n>We introduce TraceLock, a lightweight plug-in controller that instantiates this policy for a frozen diffusion language model.
• Score: 52.93186090124315
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion large language models promise faster generation by refining many token positions in parallel, but this parallelism introduces a hidden control problem: which proposed tokens should be transferred into the partially decoded sequence at each step? We refer to this decision as token commitment. Existing frozen-generator decoders largely rely on hand-designed confidence rules or block-specific acceptance filters. We argue that token commitment can instead be learned as a reusable trace-state policy. We introduce TraceLock, a lightweight plug-in controller that instantiates this policy for a frozen diffusion language model. Since oracle commitment times are unavailable, TraceLock derives self-supervision from future stability: at decoding step t, a proposed token for position i is labeled stable if it matches the final token at position i after the full decoding trace completes. The controller scores variable-length trace states and decides which active token proposals should be committed to the partially decoded sequence. Once trained for a given frozen backbone, the controller can be deployed across local-window widths, generation lengths, and step budgets without retraining or per-setting calibration. Experiments on question answering, mathematical reasoning, and code generation show that TraceLock improves the quality-step tradeoff over heuristic and learned baselines, with particularly stable behavior under cross-setting deployment. Diagnostic analyses show that its decisions are not reducible to scalar confidence, suggesting that frozen diffusion language models expose a learnable space of commitment trajectories beyond confidence-based decoding. Code is available at https://github.com/BobSun98/TraceLock.

Related papers

• Where and When to Commit: Candidate-Aware Decoding for Diffusion Language Models [29.849017661114342]
  Diffusion language models expose a provisional prediction at every denoising step.<n>Generation-time early exit stops decoding before the schedule is exhausted.<n>We introduce a training-free, candidate-aware early-exit framework that keeps the two axes separate and matches each decision to evidence of its own scope.
  arXiv  Detail & Related papers  (2026-07-30T13:04:47Z)
• Closed-Loop Dynamic Validator Node Scaling in Private Substrate Blockchains Using Takagi-Sugeno Fuzzy Inference [0.0]
  This paper presents a Takagi-Sugeno fuzzy inference system that reads live blockchain parameters.<n>It outputs a continuous efficiency score alongside a scaling recommendation: Scale Up, Maintain, or Scale Down.<n>Results show that TS fuzzy inference can support autonomous validator management in private blockchain deployments.
  arXiv  Detail & Related papers  (2026-07-08T20:26:02Z)
• Efficient and Trainable Language Model Test-Time Scaling via Local Branch Routing [65.59464371673623]
  Local Branch Routing (LBR) is a token-level test-time scaling framework.<n>It expands a small local lookahead tree, forwards all sampled branches through the language model, and uses a lightweight router to select the depth-1 subtree to commit.<n>LBR allows each token decision to use evidence beyond the root next-token distribution while avoiding full solution-level search.
  arXiv  Detail & Related papers  (2026-06-24T03:42:44Z)
• Mean-Field Parallel Decoding for Discrete Diffusion Language Models [47.086589631406035]
  We introduce a training-free decoding framework that coordinates parallel updates.<n>A variational relaxation yields a simple fixed-point update that suppresses conflicting simultaneous commitments.<n> Experiments on reasoning and code-generation benchmarks show consistent improvements in the quality-latency trade-off.
  arXiv  Detail & Related papers  (2026-06-14T13:17:58Z)
• Supportive Token Revealing for Fast Diffusion Language Model Decoding [10.022716564968695]
  AXON is a training-free module that can be added on top of existing parallel decoding strategies for diffusion language models.<n>We show that AXON improves the quality-latency trade-off of existing parallel decoders, often reducing the number of function evaluations while maintaining or improving accuracy.
  arXiv  Detail & Related papers  (2026-06-02T21:35:54Z)
• Efficient Diffusion LLMs via Temporal-Spatial Parallel Decoding and Confidence Extrapolation [15.129362213838974]
  We propose a trace-aware decoding framework with two components.<n>First, Temporal-Spatial Parallel Decoding (TSPD) uses a lightweight temporalspatial controller that consumes per-token trajectory features, including confidence, entropy, and momentum, together with token position, to decide when a token has converged and can be safely fixed.<n>Second, we introduce Confidence Extrapolation (CE), a training-free state-space module that forecasts future logit trends with uncertainty to support proactive decisions.
  arXiv  Detail & Related papers  (2026-05-29T02:29:28Z)
• When Does a Language Model Commit? A Finite-Answer Theory of Pre-Verbalization Commitment [6.48455847613423]
  Language models often generate reasoning before giving a final answer, but the visible answer does not reveal when the model's answer preference became stable.<n>We study this question through a narrow computable object: emphfinite-answer preference stabilization.<n>For a model state and specified answer verbalizers, we project the model's own continuation probabilities onto a finite answer set.<n>In binary tasks this yields an exact log-odds code, $()=S_(mathrmyesmid)-S_(mathrmnomid
  arXiv  Detail & Related papers  (2026-05-07T08:34:59Z)
• When to Commit? Towards Variable-Size Self-Contained Blocks for Discrete Diffusion Language Models [36.08108046941572]
  We propose self-containedness as a principled criterion for block commitment.<n>A block is self-contained if its predictions remain consistent with Future-Aware (FA) or without No-Future (NF) access to future context.<n>We provide theoretical justification linking self-containedness to predictive consistency, and extensive experiments validate VSB's efficacy.
  arXiv  Detail & Related papers  (2026-04-27T03:21:07Z)
• Compiling Activation Steering into Weights via Null-Space Constraints for Stealthy Backdoors [48.881343993730844]
  Safety-aligned large language models (LLMs) are increasingly deployed in real-world pipelines.<n> adversaries can distribute backdoored checkpoints that behave normally under standard evaluation but jailbreak when a hidden trigger is present.<n>Recent post-hoc weight-editing methods offer an efficient approach to injecting such backdoors.
  arXiv  Detail & Related papers  (2026-04-14T06:48:33Z)
• DIVERSED: Relaxed Speculative Decoding via Dynamic Ensemble Verification [29.426184837710952]
  Speculative decoding is an effective technique for accelerating large language model inference by drafting multiple tokens in parallel.<n>We propose Dynamic Verification Relaxed Speculative Decoding (DIVERSED), a relaxed verification framework that improves time efficiency while preserving generation quality.<n>We demonstrate that DIVERSED achieves substantially higher inference efficiency compared to standard speculative decoding methods.
  arXiv  Detail & Related papers  (2026-04-08T21:52:32Z)
• STDec: Spatio-Temporal Stability Guided Decoding for dLLMs [49.55447757907809]
  Large Diffusion Language Models (dLLMs) have achieved rapid progress, viewed as a promising alternative to the autoregressive paradigm.<n>Most dLLM decoders still adopt a global confidence threshold, and do not model local context from neighboring decoded states or temporal consistency of predicted token IDs.<n>We observe strong stability neartemporal stability in dLLM decoding, and our approach includes spatial-aware decoding and temporal-aware decoding.
  arXiv  Detail & Related papers  (2026-04-07T18:13:31Z)
• Internal Flow Signatures for Self-Checking and Refinement in LLMs [8.540143367732949]
  We introduce emphinternal flow signatures that audit decision formation from depthwise dynamics at a fixed inter-block monitoring boundary.<n>The method stabilizes token-wise motion via bias-centered monitoring, then summarizes trajectories in compact emphmoving readout-aligned subspaces.<n>A lightweight GRU validator trained on these signatures performs self-checking without modifying the base model.
  arXiv  Detail & Related papers  (2026-02-02T10:05:54Z)
• Scaling the Scaling Logic: Agentic Meta-Synthesis of Logic Reasoning [18.75349680577575]
  SSLogic is a framework for continuous family evolution with controllable difficulty.<n>Training on SSLogic-evolved data yields consistent gains over the seed baseline at matched steps.
  arXiv  Detail & Related papers  (2026-01-23T13:26:01Z)
• DiffuSpec: Unlocking Diffusion Language Models for Speculative Decoding [66.40658898418316]
  We present DiffuSpec, a training-free drop-in framework that uses a pretrained diffusion language model (DLM) to produce multi-token drafts in a single forward pass.<n>Across benchmarks, DiffuSpec yields up to 3x wall-clock speedup, establishing diffusion-based drafting as a robust alternative to autoregressive drafters for speculative decoding.
  arXiv  Detail & Related papers  (2025-09-28T07:00:15Z)
• Blockwise SFT for Diffusion Language Models: Reconciling Bidirectional Attention and Autoregressive Decoding [60.06816407728172]
  Discrete diffusion language models have shown strong potential for text generation.<n>Standard supervised fine-tuning misaligns with semi-autoregressive inference.<n>We propose Blockwise SFT, which partitions responses into fixed-size blocks.
  arXiv  Detail & Related papers  (2025-08-27T02:49:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.