CoReDiT: Spatial Coherence-Guided Token Pruning and Reconstruction for Efficient Diffusion Transformers

• URL: http://arxiv.org/abs/2605.14191v1
• Date: Wed, 13 May 2026 23:13:29 GMT
• Title: CoReDiT: Spatial Coherence-Guided Token Pruning and Reconstruction for Efficient Diffusion Transformers
• Abstract summary: Diffusion Transformers (DiTs) deliver remarkable image and video generation quality but incur high computational cost.<n>We introduce CoReDiT, a structured token pruning framework for DiTs across vision tasks.<n>CoReDiT uses a linear-time spatial coherence score to estimate local redundancy in the latent token lattice and skips high coherence (redundant) tokens in self-attention.
• Score: 47.93757224439124
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion Transformers (DiTs) deliver remarkable image and video generation quality but incur high computational cost, limiting scalability and on-device deployment. We introduce CoReDiT, a structured token pruning framework for DiTs across vision tasks. CoReDiT uses a linear-time spatial coherence score to estimate local redundancy in the latent token lattice and skips high coherence (redundant) tokens in self-attention. To maintain a dense representation and avoid visual discontinuities, we reconstruct skipped attention outputs via coherence-guided aggregation of spatially neighboring retained tokens. We further introduce a progressive, block-adaptive pruning schedule that increases pruning gradually and allocates larger budgets to blocks and denoising steps with higher redundancy. Across state-of-the-art diffusion backbones including PixArt-α and MagicDrive-V2, CoReDiT achieves up to 55% self-attention FLOPs reduction and inference speedups of 1.33x on cloud GPUs and 1.72x on mobile NPUs, while maintaining high visual quality. Notably, CoReDiT also increases on-device memory head-room, enabling higher-resolution generation.

Related papers

• Accelerating Unified Multimodal Models with Core-Expansion Routing and Unified Computation Scheduling [61.28599393707943]
  Unified multimodal models jointly support understanding and generation, but incur substantial redundant computation across tokens, layers, and generation timesteps.<n>We propose a task-shared core scorer and progress-conditioned generation expansions, optimized through generation decomposition and cross-task core alignment.<n> Experiments on two representative UMM architectures demonstrate consistent quality--efficiency improvements across both tasks, retaining 98.03% of dense understanding performance with a 1.93times$ end-to-end inference speedup.
  arXiv  Detail & Related papers  (2026-08-29T14:27:55Z)
• Stable Autoregressive Speech Generation with Low-Frame-Rate High-Dimensional Continuous Tokens [29.85404709913323]
  We show how a low-frame-rate, high-dimensional, high-bandwidth continuous representation can be co-designed with a streaming generation framework.<n>Our design preserves reconstruction quality, improves single-token predictability, achieves competitive WER, and maintains stable long-form synthesis.
  arXiv  Detail & Related papers  (2026-07-31T12:51:24Z)
• Scaling Parallel Sequence Models to Foundation-Scale Vision Encoders [123.58723804218151]
  Vision foundation models are bottlenecked by the quadratic cost of self-attention.<n>We present C-GSPN based on 2D spatial propagation propagation.
  arXiv  Detail & Related papers  (2026-05-30T14:29:43Z)
• Efficient One-Step Diffusion Restoration Model with Compact Token Compression and Linear Attention [66.63806505114263]
  Existing Real-ISR methods inherit the dense latent representations and quadratic-cost global modeling paradigm.<n>We argue that the key bottleneck lies in excessive token redundancy and costly token interactions during high-resolution restoration.<n>We introduce a linear-attention DiT with LoRA fine-tuning, enabling efficient high-resolution restoration with linear-complexity token mixing.
  arXiv  Detail & Related papers  (2026-05-22T10:07:02Z)
• TC-AE: Unlocking Token Capacity for Deep Compression Autoencoders [51.71228803075235]
  We propose TC-AE, a ViT-based architecture for deep compression autoencoders.<n>We decompose token-to-latent compression into two stages, reducing structural information loss.<n>We enhance the semantic structure of image tokens via joint self-supervised training, leading to more generative-friendly latents.
  arXiv  Detail & Related papers  (2026-04-08T17:53:52Z)
• CIAR: Interval-based Collaborative Decoding for Image Generation Acceleration [47.52310736101766]
  Auto-regressive (AR) models have recently made notable progress in image generation, achieving performance comparable to diffusion-based approaches.<n>We address this via a cloud-device collaboration framework textbfCIAR, which utilizes on-device self-verification to handle two key properties of visual synthesis.<n>Our solution centers on an on-device token uncertainty quantifier, which adopts continuous probability intervals to accelerate processing and make it feasible for large visual vocabularies.
  arXiv  Detail & Related papers  (2026-03-26T14:04:50Z)
• SFTok: Bridging the Performance Gap in Discrete Tokenizers [72.9996757048065]
  We propose textbfSFTok, a discrete tokenizer that incorporates a multi-step iterative mechanism for precise reconstruction.<n>At a high compression rate of only 64 tokens per image, SFTok achieves state-of-the-art reconstruction quality on ImageNet.
  arXiv  Detail & Related papers  (2025-12-18T18:59:04Z)
• Ultra3D: Efficient and High-Fidelity 3D Generation with Part Attention [54.15345846343084]
  We propose Ultra3D, an efficient 3D generation framework that significantly accelerates sparse voxel modeling without compromising quality.<n>Part Attention is a geometry-aware localized attention mechanism that restricts attention computation within semantically consistent part regions.<n>Experiments demonstrate that Ultra3D supports high-resolution 3D generation at 1024 resolution and achieves state-of-the-art performance in both visual fidelity and user preference.
  arXiv  Detail & Related papers  (2025-07-23T17:57:16Z)
• DnLUT: Ultra-Efficient Color Image Denoising via Channel-Aware Lookup Tables [60.95483707212802]
  DnLUT is an ultra-efficient lookup table-based framework that achieves high-quality color image denoising with minimal resource consumption.<n>Our key innovation lies in two complementary components: a Pairwise Channel Mixer (PCM) that effectively captures inter-channel correlations and spatial dependencies in parallel, and a novel L-shaped convolution design that maximizes receptive field coverage.<n>By converting these components into optimized lookup tables post-training, DnLUT achieves remarkable efficiency - requiring only 500KB storage and 0.1% energy consumption compared to its CNN contestant DnCNN, while delivering 20X faster inference.
  arXiv  Detail & Related papers  (2025-03-20T08:15:29Z)
• Faster Image2Video Generation: A Closer Look at CLIP Image Embedding's Impact on Spatio-Temporal Cross-Attentions [27.111140222002653]
  This paper investigates the role of CLIP image embeddings within the Stable Video Diffusion (SVD) framework. We introduce the VCUT, a training-free approach optimized for efficiency within the SVD architecture. The implementation of VCUT leads to a reduction of up to 322T Multiple-Accumulate Operations (MACs) per video and a decrease in model parameters by up to 50M, achieving a 20% reduction in latency compared to the baseline.
  arXiv  Detail & Related papers  (2024-07-27T08:21:14Z)
• DiTFastAttn: Attention Compression for Diffusion Transformer Models [26.095923502799664]
  Diffusion Transformers (DiT) excel at image and video generation but face computational challenges due to self-attention operators. We propose DiTFastAttn, a post-training compression method to alleviate the computational bottleneck of DiT. Our results show that for image generation, our method reduces up to 76% of the attention FLOPs and achieves up to 1.8x end-to-end speedup at high-resolution (2k x 2k) generation.
  arXiv  Detail & Related papers  (2024-06-12T18:00:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.