Reasoning to Align: Implicit Reasoning in Diffusion Transformers for Video Editing

• URL: http://arxiv.org/abs/2605.24674v1
• Date: Sat, 23 May 2026 17:22:14 GMT
• Title: Reasoning to Align: Implicit Reasoning in Diffusion Transformers for Video Editing
• Abstract summary: We propose RVEDiT, an implicit Reasoning Video Editing DiT framework built around two complementary components.<n> RVEDiT consistently outperforms state-of-the-art baselines, with particularly strong gains on localized and compositional edits.
• Score: 55.211537893248675
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Instruction-based video editing requires transforming a source video according to a natural-language instruction while preserving irrelevant content and remaining temporally coherent. We argue that existing Diffusion Transformer (DiT) editors struggle with this task for two structural reasons. First, conditioning signals are fed undifferentiated into all transformer blocks, forcing a single token stream to encode both global editing intent and fine-grained visual evidence. Second, the cross-attention patterns that govern the edit are supervised only indirectly through pixel-level reconstruction, leaving the model's internal reasoning process under-constrained. To address both limitations, we propose RVEDiT, an implicit Reasoning Video Editing DiT framework built around two complementary components. The first, Granularity-Routed Token Conditioning, introduces learnable editing tokens distilled from a multimodal LLM and routes them to shallow blocks, while reserving native visual and textual tokens for deeper blocks, thereby inducing a coarse-to-fine editing process inside the backbone. The second, Reference-Anchored Attention Alignment, employs a parameter-sharing reference branch during training and maximizes the mutual information between the attention features of the editing and reference branches, regularizing the model's internal reasoning without incurring any additional inference cost. Experiments on standard instruction-based video editing benchmarks show that RVEDiT consistently outperforms state-of-the-art baselines, with particularly strong gains on localized and compositional edits.

Related papers

• Instruction-Based Video Editing by Repurposing an Image Editing Model [55.9214783600876]
  We show that a strong instruction-based image editing model can edit videos by operating directly on video-VAE latents.<n>Our results suggest that, despite the large investment in training video latent spaces, per-frame video latents remain close enough to the image domain that mature image editing priors transfer with minimal adaptation.
  arXiv  Detail & Related papers  (2026-08-14T18:01:29Z)
• Hybrid Diffusion Transformer for Instruction-Guided Audio Editing via Rectified Flow [21.976898083812674]
  We propose a hybrid two-stage diffusion transformer architecture for instruction-guided audio editing.<n>It performs joint attention over audio and text tokens to establish coarse semantic alignment at low-resolution stage, then switches to alternating joint-attention and cross-attention blocks to refine editing details at high-resolution stage.
  arXiv  Detail & Related papers  (2026-06-18T11:20:08Z)
• SAMA: Factorized Semantic Anchoring and Motion Alignment for Instruction-Guided Video Editing [76.349958946335]
  We present SAMA (factorized Semantic Anchoring and Motion Alignment), a framework that factorizes video editing into semantic anchoring and motion modeling.<n>First, we introduce Semantic Anchoring, which establishes a reliable visual anchor by jointly predicting semantic tokens and video latents at sparse anchor frames.<n>Second, Motion Alignment pre-trains the same backbone on motion-centric video restoration pretext tasks.
  arXiv  Detail & Related papers  (2026-03-19T17:59:51Z)
• ReViSE: Towards Reason-Informed Video Editing in Unified Models with Self-Reflective Learning [57.08352504712699]
  Video unified models exhibit strong capabilities in understanding and generation, yet they struggle with reason-informed visual editing.<n>We introduce the Reason-Informed Video Editing (RVE) task, which requires reasoning about physical plausibility and causal dynamics during editing.<n>We propose ReViSE, a framework that unifies generation and evaluation within a single architecture.
  arXiv  Detail & Related papers  (2025-12-10T18:57:09Z)
• Text-Driven Reasoning Video Editing via Reinforcement Learning on Digital Twin Representations [8.479321655643195]
  We introduce reasoning video editing, a task where video editing models must interpret implicit queries through multi-hop reasoning to infer editing targets before executing modifications.<n> RIVER decouples reasoning from generation through digital twin representations of video content that preserve spatial relationships, temporal trajectories, and semantic attributes.<n> RIVER training uses reinforcement learning with rewards that evaluate reasoning accuracy and generation quality.
  arXiv  Detail & Related papers  (2025-11-18T03:37:19Z)
• FlowDirector: Training-Free Flow Steering for Precise Text-to-Video Editing [2.7248421583285265]
  FlowDirector is a novel inversion-free video editing framework.<n>Our framework models the editing process as a direct evolution in data space.<n>To achieve localized and controllable edits, we introduce an attention-guided masking mechanism.
  arXiv  Detail & Related papers  (2025-06-05T13:54:40Z)
• Re-Attentional Controllable Video Diffusion Editing [48.052781838711994]
  We propose a Re-Attentional Controllable Video Diffusion Editing (ReAtCo) method.<n>To align the spatial placement of the target objects with the edited text prompt in a training-free manner, we propose a Re-Attentional Diffusion (RAD)<n>RAD refocuses the cross-attention activation responses between the edited text prompt and the target video during the denoising stage, resulting in a spatially location-aligned and semantically high-fidelity manipulated video.
  arXiv  Detail & Related papers  (2024-12-16T12:32:21Z)
• Ground-A-Video: Zero-shot Grounded Video Editing using Text-to-image Diffusion Models [65.268245109828]
  Ground-A-Video is a video-to-video translation framework for multi-attribute video editing. It attains temporally consistent editing of input videos in a training-free manner. Experiments and applications demonstrate that Ground-A-Video's zero-shot capacity outperforms other baseline methods in terms of edit-accuracy and frame consistency.
  arXiv  Detail & Related papers  (2023-10-02T11:28:37Z)
• RIGID: Recurrent GAN Inversion and Editing of Real Face Videos [73.97520691413006]
  GAN inversion is indispensable for applying the powerful editability of GAN to real images. Existing methods invert video frames individually often leading to undesired inconsistent results over time. We propose a unified recurrent framework, named textbfRecurrent vtextbfIdeo textbfGAN textbfInversion and etextbfDiting (RIGID) Our framework learns the inherent coherence between input frames in an end-to-end manner.
  arXiv  Detail & Related papers  (2023-08-11T12:17:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.