Daily Paper Briefing

Abstract Overview

This paper proposes Hierarchical Continuous Diffusion Language Models (HC-DLM), a diffusion framework that couples discrete token generation with a continuous latent trajectory in a single reverse process. In the model, the continuous latent is the only persistent generative state, while tokens are read out from the latent at every denoising step, re-noised, and then fed back to condition the next latent update. The training objective is derived from a variational lower bound over the joint continuous-discrete trajectory, yielding reconstruction, entropy regularization, and continuous denoising terms. The paper evaluates the approach on Sudoku, Countdown, and LM1B to study structured reasoning, planning, and unconditional language generation.

Novelty

The main novelty is the hierarchical coupling between a continuous latent trajectory and per-step discrete token readouts, rather than treating discrete and continuous diffusion chains as separate sibling processes. The paper also derives a variational likelihood bound specifically for this coupled process, making the token feedback mechanism part of the generative model rather than a heuristic auxiliary signal.

Results

HC-DLM improves over matched discrete and continuous diffusion baselines across the reported tasks, though gains vary by benchmark. On Sudoku, it reaches 94.21% on the easy split and 72.41% on the hard split, outperforming the reproduced hybrid CCDD baseline on the hard split while being slightly lower on the easy split. On Countdown, the 6M-parameter HC-DLM exceeds the matched CCDD baseline on both CD4 and CD5, and on LM1B it achieves the best generative perplexity among the compared diffusion models at 75.5.

Key Points

1. HC-DLM uses a continuous latent as the only persistent state and reads out tokens at every step, feeding them back as a scaffold for subsequent denoising.
2. The method is trained through a variational objective over the coupled continuous-discrete trajectory, connecting token prediction and latent denoising in one principled framework.
3. Empirically, the model shows strong performance on globally constrained tasks and language modeling, including improved hard-split Sudoku accuracy, better 6M-scale Countdown results than the matched hybrid baseline, and the best reported diffusion-model generative perplexity on LM1B among the compared methods.

References

• arXiv: https://arxiv.org/abs/2610.02193v1
• Fugu-MT: https://fugumt.com/fugumt/paper_check/2610.02193v1
• Hugging Face Papers: https://huggingface.co/papers/2610.02193
• Project: https://hc-dlm.github.io/