ReaDiT Guidance: Control for Image and Video Generation using Diffusion Transformer Features

• URL: http://arxiv.org/abs/2609.04649v1
• Date: Fri, 04 Sep 2026 02:30:18 GMT
• Title: ReaDiT Guidance: Control for Image and Video Generation using Diffusion Transformer Features
• Abstract summary: We present DiT Readout (ReaDiT) Guidance, a framework for controlling generation with Diffusion Transformer (DiT) models.<n>ReaDiT Guidance uses features from a single DiT block to steer the generative process according to spatial targets provided at test time.
• Score: 43.21745581279017
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present DiT Readout (ReaDiT) Guidance, a lightweight framework for controlling generation with Diffusion Transformer (DiT) models via their internal feature representations. ReaDiT Guidance uses features from a single DiT block to steer the generative process according to spatial targets - like depth, pose, or edge maps - provided at test time. Furthermore, since modern text-to-video models are largely built on DiT backbones, ReaDiT Guidance naturally extends to video generation, enabling camera and motion control. Experimental results demonstrate that our approach achieves competitive or improved results compared to existing feature-based and off-the-shelf adapter-based approaches while requiring fewer parameters.

Related papers

• QWERTY: Training-Free Motion Control via Query-Warped Video Diffusion Transformers [12.871297586071364]
  Video diffusion transformers (DiTs) generate high-fidelity and temporally coherent videos, yet motion control remains implicit.<n>We introduce QWERTY, a training-free framework that enables flexible motion control in DiTs via user-defined object warping and optical flow.<n>We find that QWERTY achieves the most effective motion control among existing training-free approaches on a recent image-to-video DiT, with performance comparable to fine-tuning-based methods.
  arXiv  Detail & Related papers  (2026-07-02T08:25:38Z)
• DiTraj: training-free trajectory control for video diffusion transformer [34.05715460730871]
  Trajectory control represents a user-friendly task in controllable video generation.<n>We propose DiTraj, a training-free framework for trajectory control in text-to-video generation tailored for DiT.<n>Our method outperforms previous methods in both video quality and trajectory controllability.
  arXiv  Detail & Related papers  (2025-09-26T03:53:31Z)
• TIDE : Temporal-Aware Sparse Autoencoders for Interpretable Diffusion Transformers in Image Generation [41.909091496502704]
  Diffusion Transformers (DiTs) are a powerful yet underexplored class of generative models.<n>We propose TIDE-Temporal-aware sparse autoencoders for Interpretable Diffusion transformErs framework.
  arXiv  Detail & Related papers  (2025-03-10T08:35:51Z)
• OminiControl: Minimal and Universal Control for Diffusion Transformer [68.3243031301164]
  We present OminiControl, a novel approach that rethinks how image conditions are integrated into Diffusion Transformer (DiT) architectures.<n>OminiControl addresses these limitations through three key innovations.
  arXiv  Detail & Related papers  (2024-11-22T17:55:15Z)
• DiVE: DiT-based Video Generation with Enhanced Control [23.63288169762629]
  We propose first DiT-based framework specifically designed for generating temporally and multi-view consistent videos. Specifically, the proposed framework leverages a parameter-free spatial view-inflated attention mechanism to guarantee the cross-view consistency.
  arXiv  Detail & Related papers  (2024-09-03T04:29:59Z)
• VD3D: Taming Large Video Diffusion Transformers for 3D Camera Control [74.5434726968562]
  We show how to tame transformers video for 3D camera control using a ControlNet-like conditioning mechanism.<n>Our work is the first to enable camera control for transformer-based video diffusion models.
  arXiv  Detail & Related papers  (2024-07-17T17:59:05Z)
• Latent-Shift: Latent Diffusion with Temporal Shift for Efficient Text-to-Video Generation [115.09597127418452]
  Latent-Shift is an efficient text-to-video generation method based on a pretrained text-to-image generation model. We show that Latent-Shift achieves comparable or better results while being significantly more efficient.
  arXiv  Detail & Related papers  (2023-04-17T17:57:06Z)
• TransVOD: End-to-end Video Object Detection with Spatial-Temporal Transformers [96.981282736404]
  We present TransVOD, the first end-to-end video object detection system based on spatial-temporal Transformer architectures. Our proposed TransVOD++ sets a new state-of-the-art record in terms of accuracy on ImageNet VID with 90.0% mAP. Our proposed TransVOD Lite also achieves the best speed and accuracy trade-off with 83.7% mAP while running at around 30 FPS.
  arXiv  Detail & Related papers  (2022-01-13T16:17:34Z)
• PnP-DETR: Towards Efficient Visual Analysis with Transformers [146.55679348493587]
  Recently, DETR pioneered the solution vision tasks with transformers, it directly translates the image feature map into the object result. Recent transformer-based image recognition model andTT show consistent efficiency gain.
  arXiv  Detail & Related papers  (2021-09-15T01:10:30Z)
• End-to-End Video Object Detection with Spatial-Temporal Transformers [33.40462554784311]
  We present TransVOD, an end-to-end video object detection model based on a spatial-temporal Transformer architecture. Our method does not need complicated post-processing methods such as Seq-NMS or Tubelet rescoring. These designs boost the strong baseline deformable DETR by a significant margin (3%-4% mAP) on the ImageNet VID dataset.
  arXiv  Detail & Related papers  (2021-05-23T11:44:22Z)
• Visual Saliency Transformer [127.33678448761599]
  We develop a novel unified model based on a pure transformer, Visual Saliency Transformer (VST), for both RGB and RGB-D salient object detection (SOD) It takes image patches as inputs and leverages the transformer to propagate global contexts among image patches. Experimental results show that our model outperforms existing state-of-the-art results on both RGB and RGB-D SOD benchmark datasets.
  arXiv  Detail & Related papers  (2021-04-25T08:24:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.