dLLM: Simple Diffusion Language Modeling

• URL: http://arxiv.org/abs/2602.22661v1
• Date: Thu, 26 Feb 2026 06:26:02 GMT
• Title: dLLM: Simple Diffusion Language Modeling
• Abstract summary: dLLM is an open-source framework that unifies the core components of diffusion language modeling.<n>With dLLM, users can reproduce, finetune, deploy, and evaluate open-source large DLMs such as LLaDA and Dream.
• Score: 84.88927902712176
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Although diffusion language models (DLMs) are evolving quickly, many recent models converge on a set of shared components. These components, however, are distributed across ad-hoc research codebases or lack transparent implementations, making them difficult to reproduce or extend. As the field accelerates, there is a clear need for a unified framework that standardizes these common components while remaining flexible enough to support new methods and architectures. To address this gap, we introduce dLLM, an open-source framework that unifies the core components of diffusion language modeling -- training, inference, and evaluation -- and makes them easy to customize for new designs. With dLLM, users can reproduce, finetune, deploy, and evaluate open-source large DLMs such as LLaDA and Dream through a standardized pipeline. The framework also provides minimal, reproducible recipes for building small DLMs from scratch with accessible compute, including converting any BERT-style encoder or autoregressive LM into a DLM. We also release the checkpoints of these small DLMs to make DLMs more accessible and accelerate future research.

Related papers

• MeMo: Memory as a Model [96.54179792557863]
  Large language models (LLMs) achieve strong performance across a wide range of tasks, but remain frozen after pretraining until subsequent updates.<n>MeMo is a modular framework that encodes new knowledge into a dedicated memory model while keeping the LLM parameters unchanged.<n>MeMo achieves strong performance compared to existing methods across diverse settings.
  arXiv  Detail & Related papers  (2026-05-14T17:51:34Z)
• Locally Coherent Parallel Decoding in Diffusion Language Models [6.620088179445404]
  Diffusion language models (DLMs) offer sub-linear generation latency and bidirectional capabilities.<n>Standard DLMs sample tokens independently from conditional marginal distributions.<n>We introduce CoDiLA, a method that reconciles parallel sampling with local dependency modeling.
  arXiv  Detail & Related papers  (2026-03-03T09:56:53Z)
• A Structure-Agnostic Co-Tuning Framework for LLMs and SLMs in Cloud-Edge Systems [20.267719677908683]
  Co-PLMs is a novel framework for collaborative training of large and small language models.<n>It integrates the process of structure-agnostic mutual learning to realize knowledge exchange between the heterogeneous language models.<n>Results show that Co-PLMs outperform state-of-the-art methods, achieving average increases of 5.38% in Rouge-L and 4.88% in EM.
  arXiv  Detail & Related papers  (2025-11-12T01:16:17Z)
• Discrete Diffusion in Large Language and Multimodal Models: A Survey [61.86669998363359]
  We provide a systematic survey of Discrete Diffusion Language Models (dLLMs) and Discrete Diffusion Multimodal Language Models (dMLLMs)<n>Unlike autoregressive (AR) models, dLLMs and dMLLMs adopt a multi-token, parallel decoding paradigm using full attention and a denoising-based generation strategy.<n>We trace the historical development of dLLMs and dMLLMs, formalize the underlying mathematical frameworks, list commonly-used modeling methods, and categorize representative models.
  arXiv  Detail & Related papers  (2025-06-16T17:59:08Z)
• Large Language Models are Good Relational Learners [55.40941576497973]
  We introduce Rel-LLM, a novel architecture that utilizes a graph neural network (GNN)- based encoder to generate structured relational prompts for large language models (LLMs)<n>Unlike traditional text-based serialization approaches, our method preserves the inherent relational structure of databases while enabling LLMs to process and reason over complex entity relationships.
  arXiv  Detail & Related papers  (2025-06-06T04:07:55Z)
• LatentLLM: Attention-Aware Joint Tensor Compression [50.33925662486034]
  Large language models (LLMs) and large multi-modal models (LMMs) require a massive amount of computational and memory resources.<n>We propose a new framework to convert such LLMs/LMMs into a reduced-dimension latent structure.
  arXiv  Detail & Related papers  (2025-05-23T22:39:54Z)
• DiffLM: Controllable Synthetic Data Generation via Diffusion Language Models [38.59653405736706]
  We introduce DiffLM, a controllable data synthesis framework based on variational autoencoder (VAE)<n>We show that DiffLM generates high-quality data, with performance on downstream tasks surpassing that of real data by 2%-7% in certain cases.
  arXiv  Detail & Related papers  (2024-11-05T16:47:53Z)
• LLaVA-KD: A Framework of Distilling Multimodal Large Language Models [72.68665884790002]
  We propose a novel framework to transfer knowledge from l-MLLMs to s-MLLMs.<n>We introduce Multimodal Distillation (MDist) to transfer teacher model's robust representations across both visual and linguistic modalities.<n>We also propose a three-stage training scheme to fully exploit the potential of the proposed distillation strategy.
  arXiv  Detail & Related papers  (2024-10-21T17:41:28Z)
• Reference Trustable Decoding: A Training-Free Augmentation Paradigm for Large Language Models [79.41139393080736]
  Large language models (LLMs) have rapidly advanced and demonstrated impressive capabilities. In-Context Learning (ICL) and. Efficient Fine-Tuning (PEFT) are currently two mainstream methods for augmenting. LLMs to downstream tasks. We propose Reference Trustable Decoding (RTD), a paradigm that allows models to quickly adapt to new tasks without fine-tuning.
  arXiv  Detail & Related papers  (2024-09-30T10:48:20Z)
• Prior Knowledge Integration via LLM Encoding and Pseudo Event Regulation for Video Moment Retrieval [23.94611751368491]
  We investigate the feasibility of leveraging large language models (LLMs) for integrating general knowledge and incorporating pseudo-events as priors for temporal content distribution. To overcome these limitations, we propose utilizing LLM encoders instead of decoders. We present a general framework for integrating LLM encoders into existing VMR architectures, specifically within the fusion module.
  arXiv  Detail & Related papers  (2024-07-21T04:39:06Z)
• Exploring the Role of Large Language Models in Prompt Encoding for Diffusion Models [42.891427362223176]
  Large language models (LLMs) based on decoder-only transformers have demonstrated superior text understanding capabilities.<n>We propose a novel framework to fully harness the capabilities of LLMs.<n>We further design an LLM-Infused Diffusion Transformer (LI-DiT) based on the framework.
  arXiv  Detail & Related papers  (2024-06-17T17:59:43Z)
• ECLM: Entity Level Language Model for Spoken Language Understanding with Chain of Intent [20.3491383818583]
  Large Language Models (LLMs) have demonstrated impressive capabilities in language generation and general task performance.<n>We propose the Entity-level Language Model (ECLM) framework, which reformulates slot-filling as an entity recognition task.<n>ECLM significantly outperforms strong baselines such as Uni-MIS, achieving gains of 3.7% on MixATIS and 3.1% on MixSNIPS.
  arXiv  Detail & Related papers  (2024-03-07T13:30:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.