LogicDiff: Logic-Guided Denoising Improves Reasoning in Masked Diffusion Language Models

• URL: http://arxiv.org/abs/2603.26771v1
• Date: Tue, 24 Mar 2026 13:08:10 GMT
• Title: LogicDiff: Logic-Guided Denoising Improves Reasoning in Masked Diffusion Language Models
• Abstract summary: We introduce LogicDiff, an inference-time method that replaces confidence-based unmasking with logic-role-guided unmasking.<n>A dependency-ordered scheduler then unmasks tokens in logical dependency order.<n> LogicDiff improves LLaDA-8B-Instruct accuracy from 22.0% to 60.7% on GSM8K and from 23.6% to 23.6% on MATH-500, with less than 6% speed overhead.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Masked diffusion language models (MDLMs) generate text by iteratively unmasking tokens from a fully masked sequence, offering parallel generation and bidirectional context. However, their standard confidence-based unmasking strategy systematically defers high-entropy logical connective tokens, the critical branching points in reasoning chains, leading to severely degraded reasoning performance. We introduce LogicDiff, an inference-time method that replaces confidence-based unmasking with logic-role-guided unmasking. A lightweight classification head (4.2M parameters, 0.05% of the base model) predicts the logical role of each masked position (premise, connective, derived step, conclusion, or filler) from the base model's hidden states with 98.4% accuracy. A dependency-ordered scheduler then unmasks tokens in logical dependency order: premises first, then connectives, then derived steps, then conclusions. Without modifying a single parameter of the base model and without any reinforcement learning or task-specific training, LogicDiff improves LLaDA-8B-Instruct accuracy from 22.0% to 60.7% on GSM8K (+38.7 percentage points) and from 23.6% to 29.2% on MATH-500 (+5.6 pp), with less than 6% speed overhead. Our results demonstrate that a substantial portion of the reasoning deficit in MDLMs is attributable to suboptimal token unmasking order, not to limitations of the model's learned representations.

Related papers

• SeGDeP: Semantic- and Geometric-Aware Decoupled Prompts for Reasoning Segmentation [51.46828526392219]
  Reasoning segmentation converts an implicit linguistic conclusion into a precise mask.<n>Existing MLLM-segmenter interfaces either use a special trigger or compress both signals into one context.<n>We present SeGDeP, an explicit what-where interface.
  arXiv  Detail & Related papers  (2026-09-08T15:11:32Z)
• Mask-Aware Policy Gradients for Diffusion Language Models [24.287374086170882]
  Reinforcement learning has proven effective for improving reasoning in large language models, but extending it to Masked Diffusion Language Models (MDLMs) remains challenging.<n>We observe that MDLM generation involves two decisions at each step: what tokens to place at each masked position and which positions to remask.<n>We formalize this as a two-stage action MDP, showing that the policy gradient naturally decomposes into a token term and a masking term.
  arXiv  Detail & Related papers  (2026-07-16T16:57:34Z)
• Constrained Decoding for Diffusion Language Models via Efficient Inference over Finite Automata [57.27430779838529]
  We present an exact and tractable algorithm for sampling from the constrained mean-field posterior under any constraint expressible as a finite automaton.<n>The approach guarantees constraint satisfaction by construction, supports both greedy and sampling-based decoding, and is compatible with parallel and block-wise decoding.<n> Empirical evaluations on Dream-7B and LLaDA-8 show substantial accuracy gains across various tasks.
  arXiv  Detail & Related papers  (2026-07-08T05:48:57Z)
• Scheduling Thoughts: Learning the Order of Thought in Diffusion Language Models [51.21958617138889]
  Self-Aware Scheduling learns a lightweight order policy and applies seamlessly to both any-order and semi-autoregressive decoding.<n>On Sudoku with 1B MDM, SAS improves puzzle accuracy from 82.0% (best schedule) to 91.8%, and reaches 97.5% with second-stage fine-tuning.<n>On mathematical reasoning with LLaDA-8B, SAS improves pass@1 on GSM8K from 64% to 76% and on MBPP from 39.5% to 41%.
  arXiv  Detail & Related papers  (2026-06-22T16:32:25Z)
• JUMP: Single-Pass Membership Inference on Fine-Tuned Diffusion Language Models [5.43345665278304]
  We study membership inference attacks (MIAs) for fine-tuned discrete diffusion language models (dLLMs)<n>We propose JUMP (Joint Uncertainty-Guided Mask Probing), a single-pass scoring attack that exploits both distinctive properties of dLLMs.<n>On fine-tuned LLaDA-8B-Base across six MIMIR domains, JUMP improves mean ROC-AUC from 0.82 to 0.90 over SAMA and substantially improves low-FPR detection.
  arXiv  Detail & Related papers  (2026-05-09T05:40:00Z)
• DMax: Aggressive Parallel Decoding for dLLMs [77.24184219948337]
  We present DMax, a new paradigm for efficient diffusion language models (dLLMs)<n>It mitigates error accumulation in parallel decoding, enabling aggressive decoding while preserving generation quality.<n>At the core of our approach is On-Policy Uniform Training, a novel training strategy that efficiently unifies masked and uniform dLLMs.
  arXiv  Detail & Related papers  (2026-04-09T14:35:42Z)
• Targeted Remasking: Replacing Token Editing with Token-to-Mask Refinement in Discrete Diffusion Language Models [0.6916773850242582]
  LLaDA2.1 introduced a Token-to-Token (T2T) editing mechanism that accelerates generation.<n>We propose Token-to-Mask (T2M) remasking, a training-free, drop-in replacement for T2T editing.<n>T2M resets suspected erroneous tokens back to the mask state, allowing the diffusion process to re-predict them under cleaner context.
  arXiv  Detail & Related papers  (2026-04-06T17:39:11Z)
• Adaptation to Intrinsic Dependence in Diffusion Language Models [5.185131234265025]
  Diffusion language models (DLMs) have emerged as a promising alternative to autoregressive (AR) approaches.<n>We introduce a distribution-agnostic unmasking schedule for DLMs that adapts to the (unknown) dependence structure of the target data distribution.<n>Our results significantly improve upon prior convergence theories and yield substantial sampling acceleration for low-complexity distributions.
  arXiv  Detail & Related papers  (2026-02-23T18:41:34Z)
• Where-to-Unmask: Ground-Truth-Guided Unmasking Order Learning for Masked Diffusion Language Models [17.18632315520133]
  Masked Diffusion Language Models generate text by iteratively filling masked tokens.<n>Gt-Margin yields an oracle unmasking order that prioritizes easier positions first under each partially masked state.<n>We train a supervised unmasking planner via learning-to-rank to imitate the oracle ordering from masked contexts.
  arXiv  Detail & Related papers  (2026-02-10T07:56:46Z)
• CORE: Context-Robust Remasking for Diffusion Language Models [51.59514489363897]
  We propose Context-Robust Remasking (CORE), a training-free framework for inference-time revision.<n>Rather than trusting static token probabilities, CORE identifies context-brittle tokens by probing their sensitivity to targeted masked-context perturbations.<n>On LLaDA-8B-Base, CORE delivers consistent improvements across reasoning and code benchmarks, outperforming compute-matched baselines and improving MBPP by up to 9.2 percentage points.
  arXiv  Detail & Related papers  (2026-02-04T00:12:30Z)
• Lookahead Unmasking Elicits Accurate Decoding in Diffusion Language Models [51.12873073612084]
  Masked Diffusion Models (MDMs) as language models generate by iteratively unmasking tokens, yet their performance depends on the inference time order of unmasking.<n>We propose Lookahead Unmasking (LookUM), which addresses these concerns by reformulating sampling as path selection over all possible unmasking orders.<n>LookUM requires only two to three paths to achieve peak performance, demonstrating remarkably efficient path selection.
  arXiv  Detail & Related papers  (2025-11-04T02:37:37Z)
• Blockwise SFT for Diffusion Language Models: Reconciling Bidirectional Attention and Autoregressive Decoding [60.06816407728172]
  Discrete diffusion language models have shown strong potential for text generation.<n>Standard supervised fine-tuning misaligns with semi-autoregressive inference.<n>We propose Blockwise SFT, which partitions responses into fixed-size blocks.
  arXiv  Detail & Related papers  (2025-08-27T02:49:33Z)
• Beyond Masked and Unmasked: Discrete Diffusion Models via Partial Masking [28.55159825491572]
  Masked diffusion models (MDM) are powerful generative models for discrete data that generate samples by progressively unmasking tokens in a sequence.<n>We propose the Partial masking scheme (Prime), which augments MDM by allowing tokens to take intermediate states between the masked and unmasked states.<n>Our method demonstrates superior performance across a diverse set of generative modeling tasks.
  arXiv  Detail & Related papers  (2025-05-24T04:16:40Z)
• Bridge the Points: Graph-based Few-shot Segment Anything Semantically [79.1519244940518]
  Recent advancements in pre-training techniques have enhanced the capabilities of vision foundation models. Recent studies extend the SAM to Few-shot Semantic segmentation (FSS) We propose a simple yet effective approach based on graph analysis.
  arXiv  Detail & Related papers  (2024-10-09T15:02:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.