Controlling Motion Transfer in Diffusion Transformers via Attention Heads

• URL: http://arxiv.org/abs/2607.11081v1
• Date: Mon, 13 Jul 2026 04:44:14 GMT
• Title: Controlling Motion Transfer in Diffusion Transformers via Attention Heads
• Abstract summary: Diffusion Transformers (DiTs) have advanced video generation with high-quality, temporally coherent results.<n>We analyze video DiTs at the attention-head level and identify distinct heads specialized for motion and spatial structure.<n>We propose a head-aware controllable motion transfer framework that requires no parameter updates.
• Score: 34.45051377421395
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion Transformers (DiTs) have advanced video generation with high-quality, temporally coherent results. However, extending them to motion transfer, which requires following reference motion while aligning with a target prompt, remains challenging due to limited understanding of motion and structure representations within DiTs. We analyze video DiTs at the attention-head level and identify distinct heads specialized for motion and spatial structure. Based on this insight, we propose a head-aware controllable motion transfer framework that requires no parameter updates. Our method refines motion cues from motion-specialized heads via semantic correspondence guidance and preserves structure through selective feature injection. This head-level control not only enables accurate motion transfer but also provides an interpretable foundation for controllable video generation with DiTs.

Related papers

• QWERTY: Training-Free Motion Control via Query-Warped Video Diffusion Transformers [12.871297586071364]
  Video diffusion transformers (DiTs) generate high-fidelity and temporally coherent videos, yet motion control remains implicit.<n>We introduce QWERTY, a training-free framework that enables flexible motion control in DiTs via user-defined object warping and optical flow.<n>We find that QWERTY achieves the most effective motion control among existing training-free approaches on a recent image-to-video DiT, with performance comparable to fine-tuning-based methods.
  arXiv  Detail & Related papers  (2026-07-02T08:25:38Z)
• MotionGrounder: Grounded Multi-Object Motion Transfer via Diffusion Transformer [34.080740212072655]
  Motion transfer enables controllable video generation by transferring temporal dynamics from a reference video to synthesize a new video conditioned on a target caption.<n>We introduce MotionGrounder, a DiT-based framework that handles motion transfer with multi-object controllability.<n>Our experiments show that MotionGrounder consistently outperforms recent baselines across quantitative, qualitative, and human evaluations.
  arXiv  Detail & Related papers  (2026-04-01T13:06:03Z)
• MotionAdapter: Video Motion Transfer via Content-Aware Attention Customization [73.07309070257162]
  MotionAdapter is a content-aware motion transfer framework that enables robust and semantically aligned motion transfer.<n>Our key insight is that effective motion transfer requires explicit disentanglement of motion from appearance.<n> MotionAdapter naturally supports complex motion transfer and motion editing tasks such as zooming.
  arXiv  Detail & Related papers  (2026-01-05T10:01:27Z)
• Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidance [107.25252623824296]
  Wan-Move is a framework that brings motion control to video generative models.<n>Our core idea is to make the original condition features motion-aware for guiding video.<n>Through scaled training, Wan-Move generates 5-second, 480p videos whose motion controllability rivals Kling 1.5's commercial Motion Brush.
  arXiv  Detail & Related papers  (2025-12-09T16:13:55Z)
• MotionShot: Adaptive Motion Transfer across Arbitrary Objects for Text-to-Video Generation [23.051430600796277]
  MotionShot is a framework for parsing reference-target correspondences in a fine-grained manner.<n>It can coherently transfer motion across objects, even in the presence of significant appearance and structure disparities.
  arXiv  Detail & Related papers  (2025-07-22T07:51:05Z)
• Follow-Your-Motion: Video Motion Transfer via Efficient Spatial-Temporal Decoupled Finetuning [50.4776422843776]
  Follow-Your-Motion is an efficient two-stage video motion transfer framework.<n>We propose a spatial-temporal decoupled LoRA to decouple the attention architecture for spatial appearance and temporal motion processing.<n>During the second training stage, we design the sparse motion sampling and adaptive RoPE to accelerate the tuning speed.
  arXiv  Detail & Related papers  (2025-06-05T16:18:32Z)
• MotionPro: A Precise Motion Controller for Image-to-Video Generation [108.63100943070592]
  We present MotionPro, a precise motion controller for image-to-video (I2V) generation.<n>Region-wise trajectory and motion mask are used to regulate fine-grained motion synthesis.<n>Experiments conducted on WebVid-10M and MC-Bench demonstrate the effectiveness of MotionPro.
  arXiv  Detail & Related papers  (2025-05-26T17:59:03Z)
• Decouple and Track: Benchmarking and Improving Video Diffusion Transformers for Motion Transfer [41.26164688712492]
  Diffusion Transformers (DiT) models use 3D full attention, which does not explicitly separate temporal and spatial information.<n>We propose DeT, a method that adapts DiT models to improve motion transfer ability.<n>Our approach introduces a simple yet effective temporal kernel to smooth DiT features along the temporal dimension.<n>We also introduce explicit supervision along dense trajectories in the latent feature space to further enhance motion consistency.
  arXiv  Detail & Related papers  (2025-03-21T17:52:05Z)
• MotionAgent: Fine-grained Controllable Video Generation via Motion Field Agent [55.15697390165972]
  We propose MotionAgent, enabling fine-grained motion control for text-guided image-to-video generation.<n>The key technique is the motion field agent that converts motion information in text prompts into explicit motion fields.<n>We construct a subset of VBench to evaluate the alignment of motion information in the text and the generated video, outperforming other advanced models on motion generation accuracy.
  arXiv  Detail & Related papers  (2025-02-05T14:26:07Z)
• Spectral Motion Alignment for Video Motion Transfer using Diffusion Models [54.32923808964701]
  Spectral Motion Alignment (SMA) is a framework that refines and aligns motion vectors using Fourier and wavelet transforms.<n> SMA learns motion patterns by incorporating frequency-domain regularization, facilitating the learning of whole-frame global motion dynamics.<n>Extensive experiments demonstrate SMA's efficacy in improving motion transfer while maintaining computational efficiency and compatibility across various video customization frameworks.
  arXiv  Detail & Related papers  (2024-03-22T14:47:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.