Information-Guided Frontier Decoding: Contextual Utility-Driven Commitment in dMLLMs

• URL: http://arxiv.org/abs/2608.26641v1
• Date: Thu, 27 Aug 2026 05:51:00 GMT
• Title: Information-Guided Frontier Decoding: Contextual Utility-Driven Commitment in dMLLMs
• Abstract summary: Information-Guided Frontier Decoding is a training-free decoding strategy.<n>It ranks candidates using token confidence, neighborhood uncertainty, and structural commitment risk.<n>It encourages early commitment of reliable semantic anchors while delaying fragile structural tokens.
• Score: 5.167210035701369
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Decoding quality in diffusion multimodal language models (dMLLMs) depends heavily on the order in which masked tokens are committed. Existing confidence-based strategies prioritize locally easy tokens, but confidence does not necessarily reflect contextual usefulness. As a result, structurally easy tokens such as punctuation may be committed before informative semantic anchors, weakening context propagation and increasing error accumulation. We propose Information-Guided Frontier Decoding (IGFD), a training-free decoding strategy that ranks candidates using token confidence, neighborhood uncertainty, and structural commitment risk. IGFD encourages early commitment of reliable semantic anchors while delaying fragile structural tokens, improving contextual support during decoding. A dynamic candidate frontier further constrains token selection to locally expandable regions under the same decoding budget. The method requires no additional training, auxiliary models, or extra forward passes. Experiments across multimodal understanding, reasoning, grounding, and hallucination benchmarks show that IGFD consistently outperforms existing decoding strategies across the majority of benchmarks and diffusion MLLM backbones under identical decoding budgets.

Related papers

• Context-Aware Cluster Decoding: Semantic Anchor-Driven Coherence in dMLLMs [7.894740505622926]
  Diffusion multimodal large language models (dMLLMs) produce long-form outputs marred by semantic drift and repetition.<n>We propose ours (textbfContext-textbfAware textbfCluster textbfDecoding), a training-free decoding method that scores each masked position by a multiplicative composite of softmax confidence and neighbor proximity.
  arXiv  Detail & Related papers  (2026-08-23T11:16:04Z)
• Escaping Confidence Trap: Evolutionary Decoding for Mathematical Reasoning in Diffusion LLMs [69.52853771994451]
  We analyze the decoding trajectories of LLaDA 2.0 and identify a recurring diffusion confidence trap.<n>We propose Evolutionary Decoding, a training-free test-time scaling framework that views diffusion decoding as an evolutionary process.
  arXiv  Detail & Related papers  (2026-08-01T11:48:25Z)
• Weave of Formal Thought [51.56484100374058]
  We introduce Weave of Formal Thought (WoFT), a paradigm uniting rigorous syntactic validation with learned structural representations.<n>We present a latent-variable fine-tuning method training the language model to interleave non-terminal grammar symbols directly into generation.<n>For Python, fine-tuning StarCoder2-3B with our RWS objective reduces per-token cross-entropy by 14.3% relative to a text-only SFT baseline.
  arXiv  Detail & Related papers  (2026-06-24T15:58:11Z)
• Supportive Token Revealing for Fast Diffusion Language Model Decoding [10.022716564968695]
  AXON is a training-free module that can be added on top of existing parallel decoding strategies for diffusion language models.<n>We show that AXON improves the quality-latency trade-off of existing parallel decoders, often reducing the number of function evaluations while maintaining or improving accuracy.
  arXiv  Detail & Related papers  (2026-06-02T21:35:54Z)
• TrustLDM: Benchmarking Trustworthiness in Language Diffusion Models [41.45630253571065]
  We introduce a comprehensive trustworthiness benchmark tailored to Language Diffusion Models (LDMs)<n>We evaluate safety, privacy, and fairness across different LDM architectures with multiple categories of static post contexts.<n>Our work may potentially help the community build more trustworthy LDMs.
  arXiv  Detail & Related papers  (2026-04-15T02:19:49Z)
• CORE: Context-Robust Remasking for Diffusion Language Models [51.59514489363897]
  We propose Context-Robust Remasking (CORE), a training-free framework for inference-time revision.<n>Rather than trusting static token probabilities, CORE identifies context-brittle tokens by probing their sensitivity to targeted masked-context perturbations.<n>On LLaDA-8B-Base, CORE delivers consistent improvements across reasoning and code benchmarks, outperforming compute-matched baselines and improving MBPP by up to 9.2 percentage points.
  arXiv  Detail & Related papers  (2026-02-04T00:12:30Z)
• From Bits to Rounds: Parallel Decoding with Exploration for Diffusion Language Models [19.97248408121574]
  Diffusion Language Models (DLMs) offer comparable accuracy with faster inference speed via parallel decoding.<n>High-confidence tokens carry negligible information and strictly relying on them limits the effective progress made in each decoding round.<n>We propose Explore-Then-Exploit (ETE), a training-free decoding strategy that maximizes information throughput and decoding efficiency.
  arXiv  Detail & Related papers  (2025-11-26T06:38:37Z)
• Accelerating Diffusion LLM Inference via Local Determinism Propagation [27.751279909685604]
  LocalLeap is a training-free adaptive parallel decoding strategy.<n>It achieves 6.94$times$ throughput improvements and reduces decoding steps to just 14.2% of the original requirement.
  arXiv  Detail & Related papers  (2025-10-08T14:39:34Z)
• LayerCake: Token-Aware Contrastive Decoding within Large Language Model Layers [53.43862310647276]
  Large language models (LLMs) excel at natural language understanding and generation but remain vulnerable to factual errors.<n>We introduce a token-aware, layer-localized contrastive decoding method that aligns specific token types with their most influential transformer layers to improve factual generation.<n>Our method requires no additional training or model modification, and experiments demonstrate that our method consistently improves factuality across multiple LLMs and various benchmarks.
  arXiv  Detail & Related papers  (2025-07-06T14:35:43Z)
• Not all tokens are created equal: Perplexity Attention Weighted Networks for AI generated text detection [49.15148871877941]
  Next-token distribution outputs offer a theoretically appealing approach for detection of large language models (LLMs)<n>We propose the Perplexity Attention Weighted Network (PAWN), which uses the last hidden states of the LLM and positions to weight the sum of a series of features based on metrics from the next-token distribution across the sequence length.<n>PAWN shows competitive and even better performance in-distribution than the strongest baselines with a fraction of their trainable parameters.
  arXiv  Detail & Related papers  (2025-01-07T17:00:49Z)
• Guiding the PLMs with Semantic Anchors as Intermediate Supervision: Towards Interpretable Semantic Parsing [57.11806632758607]
  We propose to incorporate the current pretrained language models with a hierarchical decoder network. By taking the first-principle structures as the semantic anchors, we propose two novel intermediate supervision tasks. We conduct intensive experiments on several semantic parsing benchmarks and demonstrate that our approach can consistently outperform the baselines.
  arXiv  Detail & Related papers  (2022-10-04T07:27:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.