RiT: Vanilla Diffusion Transformers Suffice in Representation Space

• URL: http://arxiv.org/abs/2605.21981v1
• Date: Thu, 21 May 2026 04:21:43 GMT
• Title: RiT: Vanilla Diffusion Transformers Suffice in Representation Space
• Abstract summary: Flow matching with $x$prediction is known to exploit low-dimensional manifold structure effectively in pixel space citeli2025back.<n>We ask whether a pretrained representation space, while containing a low-dimensional data manifold of comparable intrinsic dimensionality, offers a distribution more favorable for flow-matching learning.
• Score: 12.711808725422108
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Flow matching with $x$-prediction -- regressing the clean data point rather than the ambient velocity -- is known to exploit low-dimensional manifold structure effectively in pixel space \cite{li2025back}. We ask whether a pretrained representation space, while containing a low-dimensional data manifold of comparable intrinsic dimensionality, offers a distribution more favorable for flow-matching learning. Comparing pixel, SD-VAE, and DINOv2 features along four geometric axes, we find that pixel and DINOv2 share nearly identical intrinsic dimensionalities (both $\hat{d}\!\approx\!33$) yet DINOv2 exhibits $7.3\times$ higher effective rank, $35\times$ better covariance conditioning, $11.5\times$ lower excess kurtosis, and $1.7\times$ lower on-manifold interpolation error; SD-VAE latents are consistently intermediate, indicating that the advantage stems from representation-learning objectives rather than mere compression. These statistical properties render the flow-matching regression well-conditioned and remove the need for the specialized prediction heads or Riemannian transport used by prior DINOv2 diffusion methods. We propose the \emph{Representation Image Transformer} (RiT): a vanilla Diffusion Transformer trained by $x$-prediction on frozen DINOv2 features, augmented only by a dimension-aware noise schedule and joint \texttt{[CLS]}-patch modeling. On ImageNet $256{\times}256$, RiT attains FID 1.45 without guidance and 1.14 with classifier-free guidance, outperforming DiT$^\text{DH}$-XL with $19\%$ fewer parameters (676M vs.\ 839M). The resulting ODE is efficiently solvable at coarse discretizations: with classifier-free guidance, $5$ Heun steps already reach FID 2.0 and $10$ steps reach 1.25, without distillation or consistency training. Code at https://github.com/lezhang7/RiT.

Related papers

• Round-Trip Consistency: Bidirectional Diffusion Models Can Predict Their Own Rollout Errors [51.56484100374058]
  Autoregressive models accumulate error over long rollouts, yet at deployment there is no ground truth to measure it against.<n>We train a single conditional latent diffusion model that steps a dynamical system forward or backward in time via a direction flag.<n>We show that this bidirectionality supplies a measurement-free test-time error signal.
  arXiv  Detail & Related papers  (2026-08-01T13:49:46Z)
• Stitched Value Model for Diffusion Alignment [76.52330634059528]
  Diffusion- or flow-based generative models must be aligned with task-specific rewards, such as prompt fidelity or aesthetic preference.<n>Here, we propose StitchVM, a model stitching framework that efficiently transfers reward models pretrained for clean images to the noisy latent regime.<n>We show that this approach yields improvements across a broad range of downstream steering and post-training methods.
  arXiv  Detail & Related papers  (2026-05-19T13:02:50Z)
• Finite Volume-Informed Neural Network Framework for 2D Shallow Water Equations: Rugged Loss Landscapes and the Importance of Data Guidance [3.04375043248235]
  We introduce Data-Guided FVM-PINN'', a framework that replaces the strong-form residual with a differentiable, well-balanced finite-volume (FVM) loss evaluated on unstructured meshes.<n>The framework constructs an accurate surrogate from SRH-2D anchor data, with time-window decomposition reducing error monotonically via progressive initial-condition handoff.
  arXiv  Detail & Related papers  (2026-05-09T22:03:56Z)
• Scaling Attention via Feature Sparsity [50.64995497733461]
  Scaling Transformers to ultra-long contexts is bottlenecked by the $O(n2 d)$ cost of self-attention.<n>We propose Sparse Feature Attention, where queries and keys are represented as $k$-sparse codes that preserve high-dimensional expressivity.<n>Across GPT-2 and Qwen3 pretraining, SFA matches dense baselines while improving speed by up to $2.5times$ and reducing FLOPs and KV-cache by nearly 50%.
  arXiv  Detail & Related papers  (2026-03-17T08:41:50Z)
• LoRIF: Low-Rank Influence Functions for Scalable Training Data Attribution [62.830878652285406]
  Training data attribution identifies which training examples most influenced a model's prediction.<n>LoRIF exploits low-rank structures of gradient to address both bottlenecks.<n>On models from 0.1B to 70B parameters trained on datasets with millions of examples, LoRIF achieves up to 20$times$ storage reduction and query-time speedup.
  arXiv  Detail & Related papers  (2026-01-29T16:18:34Z)
• Representation Entanglement for Generation: Training Diffusion Transformers Is Much Easier Than You Think [63.25744258438214]
  REPA and its variants effectively mitigate training challenges in diffusion models by incorporating external visual representations from pretrained models.<n>We argue that the external alignment, which is absent during the entire denoising inference process, falls short of fully harnessing the potential of discriminative representations.<n>We propose Representation Entanglement for Generation (REG), which entangles low-level image latents with a single high-level class token from pretrained foundation models for denoising.
  arXiv  Detail & Related papers  (2025-07-02T08:29:18Z)
• Simple ReFlow: Improved Techniques for Fast Flow Models [68.32300636049008]
  Diffusion and flow-matching models achieve remarkable generative performance but at the cost of many sampling steps. We propose seven improvements for training dynamics, learning and inference. We achieve state-of-the-art FID scores (without / with guidance, resp.) for fast generation via neural ODEs.
  arXiv  Detail & Related papers  (2024-10-10T11:00:55Z)
• Improved Algorithm for Adversarial Linear Mixture MDPs with Bandit Feedback and Unknown Transition [71.33787410075577]
  We study reinforcement learning with linear function approximation, unknown transition, and adversarial losses. We propose a new algorithm that attains an $widetildeO(dsqrtHS3K + sqrtHSAK)$ regret with high probability.
  arXiv  Detail & Related papers  (2024-03-07T15:03:50Z)
• Cross-view Masked Diffusion Transformers for Person Image Synthesis [21.242398582282522]
  We present X-MDPT, a novel diffusion model designed for pose-guided human image generation. X-MDPT distinguishes itself by employing masked diffusion transformers that operate on latent patches. Our model outperforms state-of-the-art approaches on the DeepFashion dataset.
  arXiv  Detail & Related papers  (2024-02-02T15:57:13Z)
• PFGM++: Unlocking the Potential of Physics-Inspired Generative Models [14.708385906024546]
  We introduce a new family of physics-inspired generative models termed PFGM++. These models realize generative trajectories for $N$ dimensional data by embedding paths in $N+D$ dimensional space. We show that models with finite $D$ can be superior to previous state-of-the-art diffusion models.
  arXiv  Detail & Related papers  (2023-02-08T18:58:02Z)
• Poisson Flow Generative Models [9.843778728210427]
  "Poisson flow" generative model maps a uniform distribution on a high-dimensional hemisphere into any data distribution. PFGM achieves current state-of-the-art performance among the normalizing flow models on CIFAR-10.
  arXiv  Detail & Related papers  (2022-09-22T17:26:58Z)
• Efficient estimation of the ANOVA mean dimension, with an application to neural net classification [0.0]
  Mean dimension of a black box function of $d$ variables is written as the sum of $d$ Sobol' indices that can be estimated by leave one out methods. We compare the variance of these leave one out methods: a Gibbs sampler called winding stairs, a radial sampler that changes each variable one at a time from a baseline, and a naive sampler that never reuses function evaluations.
  arXiv  Detail & Related papers  (2020-07-02T17:44:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.