Beyond the Bidirectional Promise: Re-evaluating the Robustness of Diffusion Language Models

• URL: http://arxiv.org/abs/2607.27386v1
• Date: Wed, 29 Jul 2026 18:46:43 GMT
• Title: Beyond the Bidirectional Promise: Re-evaluating the Robustness of Diffusion Language Models
• Abstract summary: Diffusion Language Models (DLMs) offer a compelling alternative to autoregressive (AR) generation.<n>DLMs offer a compelling alternative to autoregressive (AR) generation by enabling bidirectional context and iterative refinement.<n>We systematically evaluate DLM robustness and calibration against AR baselines.
• Score: 7.457366749109892
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion Language Models (DLMs) offer a compelling alternative to autoregressive (AR) generation by enabling bidirectional context and iterative refinement. However, their reliability under natural input noise and adversarial attacks remains under-explored. To address this, we systematically evaluate DLM robustness and calibration against AR baselines, using two parameter-matched pairs (LLaDA-8B vs. LLaMA-3-8B and Dream-7B vs. Qwen2.5-7B) across 32 natural perturbation conditions, adversarial gradient probes, and mechanistic hidden-state analyses. This paired design effectively isolates architecture-intrinsic properties from weight-dependent behaviors. We find a nuanced robustness profile: while highly stochastic DLM loss landscapes naturally resist gradient-based adversarial suffixes, they provide no guaranteed defense against natural noise, proving that everyday robustness is weight-dependent rather than inherently architectural. Furthermore, DLMs exhibit systematic overconfidence, presenting a practical deployment hazard. Most crucially, mechanistic probing reveals that all models perfectly encode input corruption, isolating behavioral fragility entirely to a decoder routing failure. Consistent with this diagnosis, we show that surface-level prompt patching fails to improve over noisy baselines. Ultimately, DLM robustness cannot be patched on; it must be fundamentally integrated into the iterative decoding loop.

Related papers

• Escaping Confidence Trap: Evolutionary Decoding for Mathematical Reasoning in Diffusion LLMs [69.52853771994451]
  We analyze the decoding trajectories of LLaDA 2.0 and identify a recurring diffusion confidence trap.<n>We propose Evolutionary Decoding, a training-free test-time scaling framework that views diffusion decoding as an evolutionary process.
  arXiv  Detail & Related papers  (2026-08-01T11:48:25Z)
• Uncertainty-aware damage identification in short-span bridges via physics-informed variational autoencoder [0.4893345190925178]
  This work proposes a physics-informed Gaussian copula variational autoencoder for structural health monitoring.<n>We replace the conventional independence assumption of latent variables with a Gaussian copula.<n>Results demonstrate that the PI-VAE accurately recovers the true posterior distribution, achieving 77.2% coverage.
  arXiv  Detail & Related papers  (2026-07-06T13:02:36Z)
• A Step Towards Robust Unsupervised Domain Adaptation via Fine-Tuning and Reinforcement Learning [2.436631469537453]
  We propose textbfSFT+RL, a two-stage robust UDA framework that integratesSupervised Fine Tuning and Reinforcement Learning.<n>textbfSFT+RL achieves average improvements of textbf10.2% in clean accuracy and textbf15.8% in adversarial robustness across three datasets.
  arXiv  Detail & Related papers  (2026-07-03T20:56:12Z)
• Hybrid Robustness Verification for Spatio-Temporal Neural Networks [19.026662893450425]
  Existing verification methods rely on overly conservative approximations or incur prohibitive computational costs.<n>In practice, adversarial perturbations exhibit structured spatial and temporal correlations, constrained to lower-dimensional, semantically meaningful subspaces.<n>We study robustness verification 3D CNNs processing video inputs, targeting applications in action recognition (UCF-101), autonomous driving (Udacity), medical imaging (MedMNIST)
  arXiv  Detail & Related papers  (2026-06-08T17:06:51Z)
• What Makes VLMs Robust? Towards Reconciling Robustness and Accuracy in Vision-Language Models [64.99746027349767]
  Adversarial Robustness Adaptation (R-Adapt) is a framework that freezes all pre-trained weights and introduces minimal, insight-driven adaptations only in the initial layers.<n>R-Adapt generalizes efficiently to large vision-language models (e.g., LLaVA and Qwen-VL) to enhance their robustness.
  arXiv  Detail & Related papers  (2026-03-13T09:02:11Z)
• Are Reasoning LLMs Robust to Interventions on Their Chain-of-Thought? [79.86483056611105]
  Reasoning LLMs generate step-by-step chains of thought before giving an answer.<n>How robust are these reasoning traces to disruptions that occur within them?<n>We introduce a controlled evaluation framework that perturbs a model's own CoT at fixed timesteps.
  arXiv  Detail & Related papers  (2026-02-07T10:02:58Z)
• Representation-Aware Unlearning via Activation Signatures: From Suppression to Knowledge-Signature Erasure [2.0017902634527194]
  We introduce Knowledge Immunization Framework (KIF), a representation-aware architecture that distinguishes genuine erasure from obfuscation.<n>Our approach combines dynamic suppression of subject-specific representations with parameter-efficient adaptation, enabling durable unlearning without full model retraining.
  arXiv  Detail & Related papers  (2026-01-15T16:28:14Z)
• Adaptive Dual Uncertainty Optimization: Boosting Monocular 3D Object Detection under Test-Time Shifts [80.32933059529135]
  Test-Time Adaptation (TTA) methods have emerged to adapt to target distributions during inference.<n>We propose Dual Uncertainty Optimization (DUO), the first TTA framework designed to jointly minimize both uncertainties for robust M3OD.<n>In parallel, we design a semantic-aware normal field constraint that preserves geometric coherence in regions with clear semantic cues.
  arXiv  Detail & Related papers  (2025-08-28T07:09:21Z)
• Learning to Align and Refine: A Foundation-to-Diffusion Framework for Occlusion-Robust Two-Hand Reconstruction [50.952228546326516]
  Two-hand reconstruction from monocular images faces persistent challenges due to complex and dynamic hand postures.<n>Existing approaches struggle with such alignment issues, often resulting in misalignment and penetration artifacts.<n>We propose a dual-stage Foundation-to-Diffusion framework that precisely align 2D prior guidance from vision foundation models.
  arXiv  Detail & Related papers  (2025-03-22T14:42:27Z)
• DRIVE: Dual-Robustness via Information Variability and Entropic Consistency in Source-Free Unsupervised Domain Adaptation [10.127634263641877]
  Adapting machine learning models to new domains without labeled data is a critical challenge in applications like medical imaging, autonomous driving, and remote sensing.<n>This task, known as Source-Free Unsupervised Domain Adaptation (SFUDA), involves adapting a pre-trained model to a target domain using only unlabeled target data.<n>Existing SFUDA methods often rely on single-model architectures, struggling with uncertainty and variability in the target domain.<n>We propose DRIVE, a novel SFUDA framework leveraging a dual-model architecture. The two models, with identical weights, work in parallel to capture diverse target domain characteristics.
  arXiv  Detail & Related papers  (2024-11-24T20:35:04Z)
• Stable Neighbor Denoising for Source-free Domain Adaptive Segmentation [91.83820250747935]
  Pseudo-label noise is mainly contained in unstable samples in which predictions of most pixels undergo significant variations during self-training. We introduce the Stable Neighbor Denoising (SND) approach, which effectively discovers highly correlated stable and unstable samples. SND consistently outperforms state-of-the-art methods in various SFUDA semantic segmentation settings.
  arXiv  Detail & Related papers  (2024-06-10T21:44:52Z)
• Efficient Adversarial Training in LLMs with Continuous Attacks [99.5882845458567]
  Large language models (LLMs) are vulnerable to adversarial attacks that can bypass their safety guardrails. We propose a fast adversarial training algorithm (C-AdvUL) composed of two losses. C-AdvIPO is an adversarial variant of IPO that does not require utility data for adversarially robust alignment.
  arXiv  Detail & Related papers  (2024-05-24T14:20:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.