Measuring Temporal Linguistic Emergence in Diffusion Language Models

• URL: http://arxiv.org/abs/2604.23235v1
• Date: Sat, 25 Apr 2026 10:17:47 GMT
• Title: Measuring Temporal Linguistic Emergence in Diffusion Language Models
• Abstract summary: Diffusion language models expose an explicit denoising trajectory.<n>We study three independent 32-step runs of LLaDA-8B-Base on WikiText-103 text.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion language models expose an explicit denoising trajectory, making it possible to ask when different kinds of information become measurable during generation. We study three independent 32-step runs of LLaDA-8B-Base on masked WikiText-103 text, each with 1{,}000 probe-training sequences and 200 held-out evaluation sequences. From saved trajectories, we derive four temporal measurements: token commitment; linear recoverability of part-of-speech (POS), coarse semantic category, and token identity; confidence and entropy dynamics; and sensitivity under mid-trajectory re-masking. Across seeds, the same ordering recurs: content categories stabilize earlier than function-heavy categories, POS and coarse semantic labels remain substantially more linearly recoverable than exact lexical identity under our probe setup, uncertainty remains higher for tokens that ultimately resolve incorrectly even though late confidence becomes less calibrated, and perturbation sensitivity peaks in the middle of the trajectory. A direct/collateral decomposition shows that this peak is overwhelmingly local to the perturbed positions themselves. In this LLaDA+WikiText setting, denoising time is therefore a useful analysis axis: under our measurements, coarse labels are recovered earlier and more robustly than lexical identity, trajectory-level uncertainty tracks eventual correctness, and mid-trajectory states are the most intervention-sensitive.

Related papers

• Dynamics of meaning: Towards the Evaluation of Diachronic Semantic Change in Sinhala [1.256381443503838]
  This study investigates the diachronic evolution of the Sinhala language from the 13th to the 20th century using a multi-stage computational framework.<n>We first align century-specific Word2Vec and FastText embeddings using Similarity Matrix Based Alignment (SMA) and Orthogonal Procrustes (OP) techniques.<n>To move beyond aggregate measures, we introduce a Bidirectional Semantic Impact Pruning approach using contextualised embeddings from a fine-tuned Llama-3.1-8B.
  arXiv  Detail & Related papers  (2026-09-08T11:43:50Z)
• Actionable Hallucination Detection: Translating Latent Uncertainty into Agentic Critique [1.1172382217477128]
  Large Language Models (LLMs) deployed as AI agents frequently exhibit user specification-grounding failures.<n>Existing detection methods fail to provide actionable, real-time correction as they either do not localize the hallucinations, or incur prohibitive inference latency.<n>We introduce the Latent Critic, a lightweight low-rank adapter (LoRA) that operates concurrently with a frozen base LLM's generation.
  arXiv  Detail & Related papers  (2026-08-11T03:26:16Z)
• Recovering Input Text from Hidden States: Study of Gradient-Based Inversion of Decoder-Only Language Models [0.0]
  We study the hidden-state inversion problem as a continuous embedding-space optimisation.<n>We analyse which tokens break the reconstructions and find a sharp categorical asymmetry.<n>Last-layer hidden states of GPT-2 are as sensitive as original text.
  arXiv  Detail & Related papers  (2026-07-01T12:18:12Z)
• Reading Calibrated Uncertainty from Language Model Trajectories [46.663987199083245]
  Methods that probe the model's internal activations feed raw hidden states into opaque snapshots and leave implicit the layer-wise trajectory by which a representation is formed.<n>We extract eleven scale-invariant geometric features, tracing the cumulative path of per-layer updates, and feed them to a sparse linear probe.<n>The probe outperforms MSP under selective abstention, with baseline scaling with up to 21 AURC points.
  arXiv  Detail & Related papers  (2026-05-19T19:24:29Z)
• Learning Uncertainty from Sequential Internal Dispersion in Large Language Models [52.29267172760918]
  Uncertainty estimation is a promising approach to detect hallucinations in large language models.<n>Recent approaches commonly depend on model internal states to estimate uncertainty.<n>We present Sequential Internal Variance Representation (SIVR), a supervised hallucination detection framework.
  arXiv  Detail & Related papers  (2026-04-17T06:31:29Z)
• HTDC: Hesitation-Triggered Differential Calibration for Mitigating Hallucination in Large Vision-Language Models [0.7252027234425333]
  Hesitation-Triggered Differential inference (HTDC) is a training-free decoding framework that preserves standard full-branch inference.<n>Hesitation-Triggered Differential inference consistently reduces hallucinations while maintaining strong task accuracy.
  arXiv  Detail & Related papers  (2026-04-13T22:47:27Z)
• Early Decisions Matter: Proximity Bias and Initial Trajectory Shaping in Non-Autoregressive Diffusion Language Models [54.16797570104461]
  Diffusion-based language models (dLLMs) have emerged as a promising alternative to autoregressive language models.<n>We investigate non-autoregressive decoding in dLLMs by systematically analyzing its inference dynamics along the temporal axis.
  arXiv  Detail & Related papers  (2026-04-12T10:26:41Z)
• OSCAR: Orchestrated Self-verification and Cross-path Refinement [7.202012136912518]
  Diffusion language models expose their denoising trajectories, offering a natural handle for inference-time control.<n>We formulate commitment uncertainty localization: given a denoising trajectory, identify token positions whose cross-chain entropy exceeds an unsupervised threshold.<n>We introduce OSCAR, a training-free inference-time framework operationalizing this formulation.
  arXiv  Detail & Related papers  (2026-04-02T05:02:22Z)
• Word-Anchored Temporal Forgery Localization [6.691985085293349]
  We propose word-anchored temporal forgery localization (WAFL) to derive candidate forgery proposals.<n>We first analyze the essence of temporal forgeries and identify the minimum meaningful forgery units, word tokens, and then align data preprocessing with the natural linguistic boundaries of speech.<n>To overcome the extreme class imbalance inherent to forgery detection, we design the artifact-centric asymmetric (ACA) loss.
  arXiv  Detail & Related papers  (2026-03-06T12:33:28Z)
• Uncertainty-Calibrated Spatiotemporal Field Diffusion with Sparse Supervision [15.213950046367538]
  We present SOLID, a mask-conditioned diffusion framework that learns from sparse observations alone.<n>Training and evaluation use only observed target locations, requiring no dense fields and no pre-imputation.
  arXiv  Detail & Related papers  (2026-02-17T16:56:55Z)
• CORE: Context-Robust Remasking for Diffusion Language Models [51.59514489363897]
  We propose Context-Robust Remasking (CORE), a training-free framework for inference-time revision.<n>Rather than trusting static token probabilities, CORE identifies context-brittle tokens by probing their sensitivity to targeted masked-context perturbations.<n>On LLaDA-8B-Base, CORE delivers consistent improvements across reasoning and code benchmarks, outperforming compute-matched baselines and improving MBPP by up to 9.2 percentage points.
  arXiv  Detail & Related papers  (2026-02-04T00:12:30Z)
• DRIFT: Detecting Representational Inconsistencies for Factual Truthfulness [5.785021425715989]
  LLMs often produce fluent but incorrect answers, yet detecting such hallucinations typically requires multiple sampling passes or post-hoc verification.<n>We propose a lightweight probe to read these signals directly from hidden states.<n>We develop an LLM router that answers confident queries immediately while delegating uncertain ones to stronger models.
  arXiv  Detail & Related papers  (2026-01-20T18:16:10Z)
• Geometric-Disentangelment Unlearning [106.99160454669902]
  gradient ascent on forget samples often harms retained knowledge.<n>We propose the Geometric-disment Unlearning (GU) that decomposes any candidate forget gradient update into tangential and normal components to retain space and executes only the normal component.<n>Our method is plug-and-play and can be attached to existing gradient-based unlearning procedures to mitigate side effects.
  arXiv  Detail & Related papers  (2025-11-21T09:58:25Z)
• STAR Loss: Reducing Semantic Ambiguity in Facial Landmark Detection [80.04000067312428]
  We propose a Self-adapTive Ambiguity Reduction (STAR) loss by exploiting the properties of semantic ambiguity. We find that semantic ambiguity results in the anisotropic predicted distribution, which inspires us to use predicted distribution to represent semantic ambiguity. We also propose two kinds of eigenvalue restriction methods that could avoid both distribution's abnormal change and the model's premature convergence.
  arXiv  Detail & Related papers  (2023-06-05T10:33:25Z)
• Weakly-Supervised Cross-Domain Adaptation for Endoscopic Lesions Segmentation [79.58311369297635]
  We propose a new weakly-supervised lesions transfer framework, which can explore transferable domain-invariant knowledge across different datasets. A Wasserstein quantified transferability framework is developed to highlight widerange transferable contextual dependencies. A novel self-supervised pseudo label generator is designed to equally provide confident pseudo pixel labels for both hard-to-transfer and easy-to-transfer target samples.
  arXiv  Detail & Related papers  (2020-12-08T02:26:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.