DiverseDiT++: Quantifying, Analyzing, and Promoting Representation Diversity in Diffusion Transformers

• URL: http://arxiv.org/abs/2608.03082v1
• Date: Tue, 04 Aug 2026 03:51:15 GMT
• Title: DiverseDiT++: Quantifying, Analyzing, and Promoting Representation Diversity in Diffusion Transformers
• Abstract summary: We show that representation diversity across blocks is a critical factor for effective representation learning in DiTs.<n>We propose DiverseDiT++, a novel framework that explicitly promotes diverse representation learning.<n>Our method incorporates long residual connections to diversify input representations across blocks and a representation diversity loss to encourage blocks to learn distinct features.
• Score: 48.96981796971673
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in Diffusion Transformers (DiTs) have enabled remarkable progress in visual synthesis, benefiting from their superior scalability. To facilitate DiTs' capability of capturing meaningful internal representations, recent works such as REPA incorporate external pretrained encoders for representation alignment. However, the underlying mechanisms governing representation learning within DiTs remain poorly understood in the community. To this end, this paper first presents a systematic analysis of the representation dynamics of DiTs via quantifying the diversity of block-wise representations. Specifically, we introduce a novel metric, termed the Weighted Diversity Score (WDS), to measure the representational discrepancies across different blocks. Through extensive investigations on the evolution and influence of internal representations under various settings, we reveal that representation diversity across blocks is a critical factor for effective representation learning in DiTs. More importantly, WDS exhibits a strong correlation with synthesis quality across diverse settings, model scales, and training stages (Pearson's $r=-0.869$ with $\log(\text{FID})$), suggesting its potential as an indicator to reflect model performance and a principled guide for model optimization. Based on this key finding, we propose DiverseDiT++, a novel framework that explicitly promotes diverse representation learning. Concretely, our method incorporates long residual connections to diversify input representations across blocks and a representation diversity loss to encourage blocks to learn distinct features. Extensive experiments on ImageNet $256\times256$ and $512\times512$ demonstrate that our DiverseDiT++ yields consistent performance gains and convergence acceleration when applied to different backbones with various sizes,...

Related papers

• ViQ: Text-Aligned Visual Quantized Representations at Any Resolution [91.46185855575789]
  We present ViQ, a Visual Quantized Representations framework, which is designed to balance semantics and details in discrete representations.<n>Our approach structures quantization learning into two stages: text-aligned pre-training and feature discretization.<n>Experiments on multimodal tasks demonstrate that ViQ achieves competitive performance compared to state-of-the-art multimodal vision encoders.
  arXiv  Detail & Related papers  (2026-06-25T17:29:27Z)
• Latent Visual States for Efficient Multimodal Reasoning [58.350643620457]
  EVA (LatEnt Visual StAtes) is a novel framework that generates continuous latent visual representations.<n>These internal representations manifest as an adaptive sequence of Latent_slot tokens.<n> EVA achieves significant performance gains while enhancing inference efficiency.
  arXiv  Detail & Related papers  (2026-06-23T07:22:22Z)
• Guiding Diffusion-based Reconstruction with Contrastive Signals for Balanced Visual Representation [81.40978077888693]
  Contrastive Language-Image Pre-training (CLIP) has become a key bottleneck for downstream performance.<n>Recent solutions use diffusion models to enhance representations by conditioning image reconstruction on CLIP visual tokens.<n>We integrate contrastive signals into diffusion-based reconstruction to pursue more comprehensive visual representations.
  arXiv  Detail & Related papers  (2026-03-05T04:45:49Z)
• DiverseDiT: Towards Diverse Representation Learning in Diffusion Transformers [18.873998808314067]
  We show that representation diversity across blocks is a crucial factor for effective learning in Diffusion Transformers (DiTs)<n>We propose DiverseDiT, a novel framework that explicitly promotes representation diversity.<n>Our work provides valuable insights into the representation learning dynamics of DiTs and offers a practical approach for enhancing their performance.
  arXiv  Detail & Related papers  (2026-03-04T16:21:42Z)
• Test-Time Conditioning with Representation-Aligned Visual Features [9.262325724962485]
  We introduce Representation-Aligned Guidance (REPA-G), a framework that leverages aligned representations with rich semantic properties.<n>We steer the denoising process toward a conditioned representation extracted from a pre-trained feature extractor.<n>Our method provides versatile control at multiple scales, ranging from fine-grained texture matching via single patches to broad semantic guidance.
  arXiv  Detail & Related papers  (2026-02-03T17:15:03Z)
• DiG: Differential Grounding for Enhancing Fine-Grained Perception in Multimodal Large Language Model [22.28268642142352]
  DiG (Differential Grounding) is a novel proxy task framework where MLLMs learn fine-grained perception by identifying and localizing all differences between similar image pairs without prior knowledge of their number.<n>Our results highlight differential grounding as a scalable and robust approach for advancing fine-grained visual reasoning in MLLMs.
  arXiv  Detail & Related papers  (2025-12-14T10:40:27Z)
• Learning Task-Agnostic Representations through Multi-Teacher Distillation [59.488314181423284]
  We introduce a task-agnostic framework based on a majority vote" objective function.<n>We demonstrate that this function is bounded by the mutual information between student and teachers' embeddings.<n>Our method effectively leverages teacher diversity, resulting in representations enabling better performance for a wide range of downstream tasks.
  arXiv  Detail & Related papers  (2025-10-21T14:36:33Z)
• Dynamic Visual Semantic Sub-Embeddings and Fast Re-Ranking [0.5242869847419834]
  We propose a Dynamic Visual Semantic Sub-Embeddings framework (DVSE) to reduce the information entropy. To encourage the generated candidate embeddings to capture various semantic variations, we construct a mixed distribution. We compare the performance with existing set-based method using four image feature encoders and two text feature encoders on three benchmark datasets.
  arXiv  Detail & Related papers  (2023-09-15T04:39:11Z)
• UniDiff: Advancing Vision-Language Models with Generative and Discriminative Learning [86.91893533388628]
  This paper presents UniDiff, a unified multi-modal model that integrates image-text contrastive learning (ITC), text-conditioned image synthesis learning (IS), and reciprocal semantic consistency modeling (RSC) UniDiff demonstrates versatility in both multi-modal understanding and generative tasks.
  arXiv  Detail & Related papers  (2023-06-01T15:39:38Z)
• Demystify Transformers & Convolutions in Modern Image Deep Networks [80.16624587948368]
  This paper aims to identify the real gains of popular convolution and attention operators through a detailed study.<n>We find that the key difference among these feature transformation modules, such as attention or convolution, lies in their spatial feature aggregation approach.<n>Various STMs are integrated into this unified framework for comprehensive comparative analysis.
  arXiv  Detail & Related papers  (2022-11-10T18:59:43Z)
• Investigating the Properties of Neural Network Representations in Reinforcement Learning [35.02223992335008]
  This paper empirically investigates the properties of representations that support transfer in reinforcement learning. We consider Deep Q-learning agents with different auxiliary losses in a pixel-based navigation environment. We develop a method to better understand why some representations work better for transfer, through a systematic approach.
  arXiv  Detail & Related papers  (2022-03-30T00:14:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.