Don't Commit Alone: Joint Token Commitment in Diffusion Large Language Models

• URL: http://arxiv.org/abs/2607.04469v1
• Date: Sun, 05 Jul 2026 19:29:13 GMT
• Title: Don't Commit Alone: Joint Token Commitment in Diffusion Large Language Models
• Abstract summary: We propose CoCommit, a marker-gated coordination pass that briefly defers commitment.<n>On LLaDA2.1-mini with LoRA adapters and matched greedy inference, joint commitment improves accuracy on all six benchmarks we evaluate.
• Score: 0.6916773850242582
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion large language models (dLLMs) commit multiple tokens per denoising step by decoding each selected position independently from the shared context; when those positions are dependent, the resulting factorization error is captured by conditional total correlation, which confidence-based selection cannot observe from marginals alone. We propose CoCommit, a marker-gated coordination pass that briefly defers commitment: after the usual bundle selection, a learned marker announces the commit set and the backbone's last-$n$ layers are re-applied so marked positions coordinate -- approximating joint-mode decoding -- before greedy argmax writes tokens. The method reuses existing weights with one extra partial forward pass and no auxiliary model. On LLaDA2.1-mini with LoRA adapters and matched greedy inference, joint commitment improves accuracy on all six benchmarks we evaluate, with the largest gains on reasoning and exact-answer tasks.

Related papers

• Neither Parallel Nor Sequential: How DiffusionGemma Actually Commits Tokens [0.08599681538174887]
  Open diffusion language models are marketed as parallel, non-autoregressive decoders.<n>We instrument a masked discrete-diffusion mixture-of-experts model built on Gemma 4.<n>Across a 686-prompt, six-regime probe suite we find that its decoding is neither parallel nor block-autoregressive.
  arXiv  Detail & Related papers  (2026-06-12T16:45:11Z)
• Attention-Discounted Adaptive Sampler for Masked Diffusion Language Models [59.51249894128724]
  Masked diffusion language models can reduce inference steps by revealing multiple tokens per denoising iteration.<n>We propose ADAS, a training-free reranking rule for parallel masked diffusion decoding.
  arXiv  Detail & Related papers  (2026-06-09T13:17:27Z)
• Understanding-Enhanced Model Collaboration for Long-Tailed Egocentric Mistake Detection [85.49213290363834]
  We propose an Understanding-Enhanced Model Collaboration Method (UE-MCM)<n>It combines efficient coarse-grained video understanding with accurate fine-grained action reasoning.<n>The resulting system balances speed and accuracy, making it effective for detecting subtle, rare, and ambiguous mistakes in egocentric instructional videos.
  arXiv  Detail & Related papers  (2026-06-01T11:50:16Z)
• Cluster-Level Attention-Guided Parallel Decoding for Masked Diffusion Language Models [13.325071163425621]
  Masked diffusion language models (MDLMs) enable parallel decoding by predicting all masked positions at each denoising step.<n>We revisit this granularity and observe that reliable predictions often emerge as contiguous high-confidence spans.<n>We use self-attention maps to estimate inter-cluster dependencies, enabling conflict-aware selection of mutually compatible CICs for parallel commitment.
  arXiv  Detail & Related papers  (2026-05-28T08:42:39Z)
• Context Pruning for Coding Agents via Multi-Rubric Latent Reasoning [14.508049757681862]
  LLM-powered coding agents spend the majority of their token budget reading repository files.<n>We show that this formulation creates a modeling bottleneck.<n>We propose LaMR, a structured pruning framework that decomposes code relevance into two interpretable quality dimensions.
  arXiv  Detail & Related papers  (2026-05-14T18:30:10Z)
• Dependency-Guided Parallel Decoding in Discrete Diffusion Language Models [21.699371484195865]
  We propose DEMASK, a lightweight dependency predictor that attaches to the final hidden states of a dLLM.<n> Empirically, DEMASK achieves 1.7-2.2$times$ speedup on Dream-7B while matching or improving accuracy compared to confidence-based and KL-based baselines.
  arXiv  Detail & Related papers  (2026-04-02T22:21:24Z)
• Label-Free Cross-Task LoRA Merging with Null-Space Compression [50.63908869296697]
  We introduce Null-Space Compression (NSC) Merging, a label-free, output-agnostic method that sets merge weights from adapter geometry.<n>NSC achieves state-of-the-art performance across twenty heterogeneous vision tasks with balanced gains where prior methods overfit subsets of tasks.
  arXiv  Detail & Related papers  (2026-03-27T11:34:41Z)
• Not all tokens are created equal: Perplexity Attention Weighted Networks for AI generated text detection [49.15148871877941]
  Next-token distribution outputs offer a theoretically appealing approach for detection of large language models (LLMs)<n>We propose the Perplexity Attention Weighted Network (PAWN), which uses the last hidden states of the LLM and positions to weight the sum of a series of features based on metrics from the next-token distribution across the sequence length.<n>PAWN shows competitive and even better performance in-distribution than the strongest baselines with a fraction of their trainable parameters.
  arXiv  Detail & Related papers  (2025-01-07T17:00:49Z)
• Localizing Factual Inconsistencies in Attributable Text Generation [74.11403803488643]
  We introduce QASemConsistency, a new formalism for localizing factual inconsistencies in attributable text generation.<n>We show that QASemConsistency yields factual consistency scores that correlate well with human judgments.
  arXiv  Detail & Related papers  (2024-10-09T22:53:48Z)
• Object Segmentation by Mining Cross-Modal Semantics [68.88086621181628]
  We propose a novel approach by mining the Cross-Modal Semantics to guide the fusion and decoding of multimodal features. Specifically, we propose a novel network, termed XMSNet, consisting of (1) all-round attentive fusion (AF), (2) coarse-to-fine decoder (CFD), and (3) cross-layer self-supervision.
  arXiv  Detail & Related papers  (2023-05-17T14:30:11Z)
• Consensus-Guided Correspondence Denoising [67.35345850146393]
  We propose to denoise correspondences with a local-to-global consensus learning framework to robustly identify correspondence. A novel "pruning" block is introduced to distill reliable candidates from initial matches according to their consensus scores estimated by dynamic graphs from local to global regions. Our method outperforms state-of-the-arts on robust line fitting, wide-baseline image matching and image localization benchmarks by noticeable margins.
  arXiv  Detail & Related papers  (2021-01-03T09:10:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.