TeDiO: Temporal Diagonal Optimization for Training-Free Coherent Video Diffusion

• URL: http://arxiv.org/abs/2605.14136v1
• Date: Wed, 13 May 2026 21:39:50 GMT
• Title: TeDiO: Temporal Diagonal Optimization for Training-Free Coherent Video Diffusion
• Abstract summary: We show that incoherent videos consistently exhibit irregular, fragmented temporal diagonals in their intermediate self-attention maps.<n>We introduce TeDiO, a training-free, inference-time method that reinforces temporal consistency by regularizing these internal attention patterns.
• Score: 30.705936075333465
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent text-to-video diffusion transformers generate visually compelling frames, yet still struggle with temporal coherence, often producing flickering, drifting, or unstable motion. We show that these failures leave a clear imprint inside the model: incoherent videos consistently exhibit irregular, fragmented temporal diagonals in their intermediate self-attention maps, whereas stable motion corresponds to smooth, band-diagonal patterns. Building on this observation, we introduce TeDiO, a training-free, inference-time method that reinforces temporal consistency by regularizing these internal attention patterns. TeDiO estimates diagonal smoothness, identifies unstable regions, and performs lightweight latent updates that promote coherent frame-to-frame dynamics, without modifying model weights or using external motion supervision. Across multiple video diffusion models (e.g., Wan2.1, CogVideoX), TeDiO delivers markedly smoother motion while preserving per-frame visual quality, offering an efficient plug-and-play approach to improving dynamic realism in modern video generation systems.

Related papers

• SNM-VFI: Symmetric Nonlinear Motion-Guided Generative Video Frame Interpolation [54.26896703895789]
  We propose a training-free framework for motion-controllable generative video frames with pre-trained optical flow and video diffusion models.<n>SNM-VFI achieves strong perceptual quality, competitive reconstruction accuracy, and robust temporal coherence across diverse motion scenarios.
  arXiv  Detail & Related papers  (2026-08-13T16:43:49Z)
• Bridging Event Streams and DiT: Event-Guided Video Frame Interpolation [75.95737622994548]
  Event cameras provide high-resolution motion cues that are well suited for bridging temporal gaps.<n>We propose an adapter-based framework that incorporates event-derived cues into a pre-trained image-to-video diffusion model.
  arXiv  Detail & Related papers  (2026-08-11T04:47:44Z)
• DRFusion: Drift-Resilient Temporally Consistent Infrared-Visible Video Fusion [21.262410801452948]
  Infrared and visible video fusion is essential for achieving comprehensive perception in dynamic scenes.<n>Maintaining temporal consistency remains a formidable challenge.<n>We propose a drift-resilient video fusion method that reformulates the task as history-conditioned motion generation.
  arXiv  Detail & Related papers  (2026-05-25T12:23:55Z)
• SHIFT: Motion Alignment in Video Diffusion Models with Adversarial Hybrid Fine-Tuning [47.36837823291006]
  We study the problem of motion alignment in video diffusion models post-training. To address this, we introduce pixel-motion rewards based on pixel flux dynamics.<n>We propose Smooth Hybrid Fine-tuning (SHIFT), a scalable reward-driven fine-tuning framework for video diffusion models.
  arXiv  Detail & Related papers  (2026-03-18T07:04:02Z)
• Knot Forcing: Taming Autoregressive Video Diffusion Models for Real-time Infinite Interactive Portrait Animation [16.692450893925148]
  We present a novel streaming framework named Knot Forcing for real-time portrait animation.<n>K Knot Forcing enables high-fidelity, temporally consistent, and interactive portrait animation over infinite sequences.
  arXiv  Detail & Related papers  (2025-12-25T16:34:56Z)
• Zero-Shot Video Translation and Editing with Frame Spatial-Temporal Correspondence [81.82643953694485]
  We present FRESCO, which integrates intra-frame correspondence with inter-frame correspondence to formulate a more robust spatial-temporal constraint.<n>Our method goes beyond attention guidance to explicitly optimize features, achieving high spatial-temporal consistency with the input video.<n>We verify FRESCO adaptations on two zero-shot tasks of video-to-video translation and text-guided video editing.
  arXiv  Detail & Related papers  (2025-12-03T15:51:11Z)
• Tracktention: Leveraging Point Tracking to Attend Videos Faster and Better [61.381599921020175]
  Temporal consistency is critical in video prediction to ensure that outputs are coherent and free of artifacts.<n>Traditional methods, such as temporal attention and 3D convolution, may struggle with significant object motion.<n>We propose the Tracktention Layer, a novel architectural component that explicitly integrates motion information using point tracks.
  arXiv  Detail & Related papers  (2025-03-25T17:58:48Z)
• ByTheWay: Boost Your Text-to-Video Generation Model to Higher Quality in a Training-free Way [72.1984861448374]
  ByTheWay is a training-free method to improve the quality of text-to-video generation without introducing additional parameters, augmenting memory or sampling time.<n>It improves structural plausibility and temporal consistency of generated videos by reducing the disparity between the temporal attention maps across various decoder blocks.<n>It enhances the magnitude and richness of motion by amplifying the energy of the map.
  arXiv  Detail & Related papers  (2024-10-08T17:56:33Z)
• Live2Diff: Live Stream Translation via Uni-directional Attention in Video Diffusion Models [64.2445487645478]
  Large Language Models have shown remarkable efficacy in generating streaming data such as text and audio. We present Live2Diff, the first attempt at designing a video diffusion model with uni-directional temporal attention, specifically targeting live streaming video translation.
  arXiv  Detail & Related papers  (2024-07-11T17:34:51Z)
• FRESCO: Spatial-Temporal Correspondence for Zero-Shot Video Translation [85.29772293776395]
  We introduce FRESCO, intra-frame correspondence alongside inter-frame correspondence to establish a more robust spatial-temporal constraint. This enhancement ensures a more consistent transformation of semantically similar content across frames. Our approach involves an explicit update of features to achieve high spatial-temporal consistency with the input video.
  arXiv  Detail & Related papers  (2024-03-19T17:59:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.