Subliminal Clocks: Latent Time Modelling in Diffusion Language Models

• URL: http://arxiv.org/abs/2607.01774v1
• Date: Thu, 02 Jul 2026 06:45:42 GMT
• Title: Subliminal Clocks: Latent Time Modelling in Diffusion Language Models
• Abstract summary: Diffusion Language Models (DLMs) have emerged as a promising alternative to autoregressive models.<n>We show that DLMs do in fact encode a latent representation related to the diffusion timestep within their residual streams.<n>We further demonstrate that steering the model along a low-dimensional subspace associated with the inferred timestep allows us to systematically modulate its notion of denoising progress.
• Score: 19.412917551000742
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Diffusion Language Models (DLMs) have recently emerged as a promising alternative to autoregressive models. Unlike standard diffusion-based approaches, DLMs are not explicitly conditioned on a timestep, raising a natural question: do these models internally represent denoising progress, and how is such information used downstream? In this work, we show that DLMs do in fact encode a latent representation related to the diffusion timestep within their residual streams. We find that this signal can be reliably extracted using probes across layers, indicating that denoising progress is decodable from internal activations. We further demonstrate that steering the model along a low-dimensional subspace associated with the inferred timestep allows us to systematically modulate its notion of denoising progress, leading to predictable changes in model confidence and entropy. Finally, we analyse the geometry of the identified representation, showing that it exhibits structured and interpretable properties in activation space, and shedding light on how such a signal is processed by these models.

Related papers

• Learned Relay Representations for Forward-Thinking Discrete Diffusion Models [34.17541648016911]
  We propose Learned Relay Representations (Relay) to avoid a hard reset between denoising rounds.<n>Relay introduces a differentiable per-token channel that passes information between forward passes and is trained via truncated backpropagation through time.<n>We show that Relay can be scaled to state-of-the-art Diffusion Language Models (DLMs) and is seamlessly compatible with techniques like block diffusion and KV caching.
  arXiv  Detail & Related papers  (2026-05-21T18:53:22Z)
• One-step Language Modeling via Continuous Denoising [36.18484491074519]
  We show that language models leveraging flow-based continuous denoising can outperform discrete diffusion in both quality and speed.<n>Our work calls into question the widely held hypothesis that discrete diffusion processes are necessary for generative modeling over discrete modalities.
  arXiv  Detail & Related papers  (2026-02-18T19:23:07Z)
• The Entropic Signature of Class Speciation in Diffusion Models [12.212582407125089]
  We show that tracking the class-conditional entropy of a latent semantic variable given a noisy state provides a reliable signature of transition regimes.<n>We validate our method on EDM2-XS and Stable Diffusion 1.5, where class-conditional entropy consistently isolates the noise regimes critical for semantic structure formation.
  arXiv  Detail & Related papers  (2026-02-10T10:56:46Z)
• Continuously Augmented Discrete Diffusion model for Categorical Generative Modeling [87.34677262370924]
  Standard discrete diffusion models treat all unobserved states identically by mapping them to an absorbing [MASK] token.<n>This creates an 'information void' where semantic information that could be inferred from unmasked tokens is lost between denoising steps.<n>We introduce Continuously Augmented Discrete Diffusion, a framework that augments the discrete state space with a paired diffusion in a continuous latent space.
  arXiv  Detail & Related papers  (2025-10-01T18:00:56Z)
• FlowMo: Variance-Based Flow Guidance for Coherent Motion in Video Generation [51.110607281391154]
  FlowMo is a training-free guidance method for enhancing motion coherence in text-to-video models.<n>It estimates motion coherence by measuring the patch-wise variance across the temporal dimension and guides the model to reduce this variance dynamically during sampling.
  arXiv  Detail & Related papers  (2025-06-01T19:55:33Z)
• Simple and Critical Iterative Denoising: A Recasting of Discrete Diffusion in Graph Generation [0.0]
  dependencies between intermediate noisy states lead to error accumulation and propagation during the reverse denoising process.<n>We propose a novel framework called Simple Iterative Denoising, which simplifies discrete diffusion and circumvents the issue.<n>Our empirical evaluations demonstrate that the proposed method significantly outperforms existing discrete diffusion baselines in graph generation tasks.
  arXiv  Detail & Related papers  (2025-03-27T15:08:58Z)
• Dynamical Diffusion: Learning Temporal Dynamics with Diffusion Models [71.63194926457119]
  We introduce Dynamical Diffusion (DyDiff), a theoretically sound framework that incorporates temporally aware forward and reverse processes.<n>Experiments across scientifictemporal forecasting, video prediction, and time series forecasting demonstrate that Dynamical Diffusion consistently improves performance in temporal predictive tasks.
  arXiv  Detail & Related papers  (2025-03-02T16:10:32Z)
• Continuous Diffusion Model for Language Modeling [64.7425225935854]
  Existing continuous diffusion models for discrete data underperform compared to discrete methods.<n>We propose a continuous diffusion model for language modeling that incorporates the geometry of the underlying categorical distribution.<n>Our method outperforms existing discrete diffusion models and approaches the performance of autoregressive models.
  arXiv  Detail & Related papers  (2025-02-17T08:54:29Z)
• Latent diffusion models for parameterization and data assimilation of facies-based geomodels [0.0]
  Diffusion models are trained to generate new geological realizations from input fields characterized by random noise. Latent diffusion models are shown to provide realizations that are visually consistent with samples from geomodeling software.
  arXiv  Detail & Related papers  (2024-06-21T01:32:03Z)
• ShiftDDPMs: Exploring Conditional Diffusion Models by Shifting Diffusion Trajectories [144.03939123870416]
  We propose a novel conditional diffusion model by introducing conditions into the forward process. We use extra latent space to allocate an exclusive diffusion trajectory for each condition based on some shifting rules. We formulate our method, which we call textbfShiftDDPMs, and provide a unified point of view on existing related methods.
  arXiv  Detail & Related papers  (2023-02-05T12:48:21Z)
• Discrete Denoising Flows [87.44537620217673]
  We introduce a new discrete flow-based model for categorical random variables: Discrete Denoising Flows (DDFs) In contrast with other discrete flow-based models, our model can be locally trained without introducing gradient bias. We show that DDFs outperform Discrete Flows on modeling a toy example, binary MNIST and Cityscapes segmentation maps, measured in log-likelihood.
  arXiv  Detail & Related papers  (2021-07-24T14:47:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.