Dynamic Chunking Diffusion Transformer

• URL: http://arxiv.org/abs/2603.06351v1
• Date: Fri, 06 Mar 2026 14:59:11 GMT
• Title: Dynamic Chunking Diffusion Transformer
• Abstract summary: Diffusion Transformers process images as fixed-length sequences of tokens produced by a static $textitpatchify$ operation.<n>We introduce the Dynamic Chunking Diffusion Transformer (DC-DiT), which augments the DiT backbone with a learned encoder-router-decoder scaffold.<n>DC-DiT learns to compress uniform background regions into fewer tokens and detail-rich regions into more tokens, with meaningful visual segmentations emerging without explicit supervision.
• Score: 16.954365273223473
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Diffusion Transformers process images as fixed-length sequences of tokens produced by a static $\textit{patchify}$ operation. While effective, this design spends uniform compute on low- and high-information regions alike, ignoring that images contain regions of varying detail and that the denoising process progresses from coarse structure at early timesteps to fine detail at late timesteps. We introduce the Dynamic Chunking Diffusion Transformer (DC-DiT), which augments the DiT backbone with a learned encoder-router-decoder scaffold that adaptively compresses the 2D input into a shorter token sequence in a data-dependent manner using a chunking mechanism learned end-to-end with diffusion training. The mechanism learns to compress uniform background regions into fewer tokens and detail-rich regions into more tokens, with meaningful visual segmentations emerging without explicit supervision. Furthermore, it also learns to adapt its compression across diffusion timesteps, using fewer tokens at noisy stages and more tokens as fine details emerge. On class-conditional ImageNet $256{\times}256$, DC-DiT consistently improves FID and Inception Score over both parameter-matched and FLOP-matched DiT baselines across $4{\times}$ and $16{\times}$ compression, showing this is a promising technique with potential further applications to pixel-space, video and 3D generation. Beyond accuracy, DC-DiT is practical: it can be upcycled from pretrained DiT checkpoints with minimal post-training compute (up to $8{\times}$ fewer training steps) and composes with other dynamic computation methods to further reduce generation FLOPs.

Related papers

• One Model, Many Budgets: Elastic Latent Interfaces for Diffusion Transformers [80.19461768457622]
  Elastic Latent Interface Transformer (ELIT) is a drop-in, DiT-compatible mechanism that decouples input image size from compute.<n>Read and Write cross-attention layers move information between spatial tokens and latents and prioritize important input regions.<n>On ImageNet-1K 512px, ELIT delivers an average gain of $35.3%$ and $39.6%$ in FID and FDD scores.
  arXiv  Detail & Related papers  (2026-03-12T17:57:04Z)
• Sprint: Sparse-Dense Residual Fusion for Efficient Diffusion Transformers [91.02299679350834]
  Diffusion Transformers (DiTs) deliver state-of-the-art generative performance but their quadratic training cost with sequence length makes large-scale pretraining prohibitively expensive.<n>We present Sparse--Dense Residual Fusion for Efficient Diffusion Transformers, a simple method that enables aggressive token dropping (up to 75%) while preserving quality.
  arXiv  Detail & Related papers  (2025-10-24T19:29:55Z)
• Representation Entanglement for Generation: Training Diffusion Transformers Is Much Easier Than You Think [63.25744258438214]
  REPA and its variants effectively mitigate training challenges in diffusion models by incorporating external visual representations from pretrained models.<n>We argue that the external alignment, which is absent during the entire denoising inference process, falls short of fully harnessing the potential of discriminative representations.<n>We propose Representation Entanglement for Generation (REG), which entangles low-level image latents with a single high-level class token from pretrained foundation models for denoising.
  arXiv  Detail & Related papers  (2025-07-02T08:29:18Z)
• Attend to Not Attended: Structure-then-Detail Token Merging for Post-training DiT Acceleration [24.85624444212476]
  This study proposes a novel concept: attend to prune feature redundancies in areas not attended by the diffusion process.<n>We analyze the location and degree of feature redundancies based on the structure-then-detail denoising priors.<n>We introduce SDTM, a structure-then-detail token merging approach that dynamically compresses feature redundancies.
  arXiv  Detail & Related papers  (2025-05-16T21:27:38Z)
• DDT: Decoupled Diffusion Transformer [51.84206763079382]
  Diffusion transformers encode noisy inputs to extract semantic component and decode higher frequency with identical modules.<n>textbfcolorddtDecoupled textbfcolorddtTransformer(textbfcolorddtDDT)<n>textbfcolorddtTransformer(textbfcolorddtDDT)<n>textbfcolorddtTransformer(textbfcolorddtDDT)
  arXiv  Detail & Related papers  (2025-04-08T07:17:45Z)
• CODA: Repurposing Continuous VAEs for Discrete Tokenization [31.932323809073477]
  textbfCODA(textbfCOntinuous-to-textbfDiscrete textbfAdaptation) is a framework that decouples compression and discretization.<n>Our approach achieves a remarkable codebook utilization of 100% and notable reconstruction FID (rFID) of $mathbf0.43$ and $mathbf1.34$ for $8 times$ and $16 times$ compression on ImageNet 256$times$ 256 benchmark.
  arXiv  Detail & Related papers  (2025-03-22T12:59:00Z)
• Layer- and Timestep-Adaptive Differentiable Token Compression Ratios for Efficient Diffusion Transformers [55.87192133758051]
  Diffusion Transformers (DiTs) have achieved state-of-the-art (SOTA) image generation quality but suffer from high latency and memory inefficiency.<n>We propose DiffCR, a dynamic DiT inference framework with differentiable compression ratios.
  arXiv  Detail & Related papers  (2024-12-22T02:04:17Z)
• Transformer based Pluralistic Image Completion with Reduced Information Loss [72.92754600354199]
  Transformer based methods have achieved great success in image inpainting recently. They regard each pixel as a token, thus suffering from an information loss issue. We propose a new transformer based framework called "PUT"
  arXiv  Detail & Related papers  (2024-03-31T01:20:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.