Video Consistency Distance: Enhancing Temporal Consistency for Image-to-Video Generation via Reward-Based Fine-Tuning

• URL: http://arxiv.org/abs/2510.19193v2
• Date: Thu, 23 Oct 2025 07:07:25 GMT
• Title: Video Consistency Distance: Enhancing Temporal Consistency for Image-to-Video Generation via Reward-Based Fine-Tuning
• Authors: Takehiro Aoshima, Yusuke Shinohara, Byeongseon Park,
• Abstract summary: Reward-based fine-tuning of video diffusion models is an effective approach to improve the quality of generated videos.<n>We propose Video Consistency Distance (VCD), a novel metric designed to enhance temporal consistency.
• Score: 5.847416016271551
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reward-based fine-tuning of video diffusion models is an effective approach to improve the quality of generated videos, as it can fine-tune models without requiring real-world video datasets. However, it can sometimes be limited to specific performances because conventional reward functions are mainly aimed at enhancing the quality across the whole generated video sequence, such as aesthetic appeal and overall consistency. Notably, the temporal consistency of the generated video often suffers when applying previous approaches to image-to-video (I2V) generation tasks. To address this limitation, we propose Video Consistency Distance (VCD), a novel metric designed to enhance temporal consistency, and fine-tune a model with the reward-based fine-tuning framework. To achieve coherent temporal consistency relative to a conditioning image, VCD is defined in the frequency space of video frame features to capture frame information effectively through frequency-domain analysis. Experimental results across multiple I2V datasets demonstrate that fine-tuning a video generation model with VCD significantly enhances temporal consistency without degrading other performance compared to the previous method.

Related papers

• Towards Holistic Modeling for Video Frame Interpolation with Auto-regressive Diffusion Transformers [95.68243351895107]
  We propose a holistic, video-centric paradigm named textbfLocal textbfDiffusion textbfForcing for textbfVideo textbfFrame textbfInterpolation (LDF-VFI)<n>Our framework is built upon an auto-regressive diffusion transformer that models the entire video sequence to ensure long-range temporal coherence.<n>LDF-VFI achieves state-of-the-art performance on challenging long-sequence benchmarks, demonstrating superior per
  arXiv  Detail & Related papers  (2026-01-21T12:58:52Z)
• Autoregressive Video Autoencoder with Decoupled Temporal and Spatial Context [8.458436768725212]
  Video autoencoders compress videos into compact latent representations for efficient reconstruction.<n>We propose Autoregressive Video Autoencoder (ARVAE), which compresses and reconstructs each frame conditioned on its predecessor in an autoregressive manner.<n>ARVAE achieves superior reconstruction quality with extremely lightweight models and small-scale training data.
  arXiv  Detail & Related papers  (2025-12-12T05:40:01Z)
• STCDiT: Spatio-Temporally Consistent Diffusion Transformer for High-Quality Video Super-Resolution [60.06664986365803]
  We present STCDiT, a video super-resolution framework built upon a pre-trained video diffusion model.<n>It aims to restore structurally faithful and temporally stable videos from degraded inputs, even under complex camera motions.
  arXiv  Detail & Related papers  (2025-11-24T05:37:23Z)
• STR-Match: Matching SpatioTemporal Relevance Score for Training-Free Video Editing [35.50656689789427]
  STR-Match is a training-free video editing system that produces visually appealing and coherent videos.<n> STR-Match consistently outperforms existing methods in both visual quality andtemporal consistency.
  arXiv  Detail & Related papers  (2025-06-28T12:36:19Z)
• Redefining Temporal Modeling in Video Diffusion: The Vectorized Timestep Approach [29.753974393652356]
  We propose a frame-aware video diffusion model(FVDM) Our approach allows each frame to follow an independent noise schedule, enhancing the model's capacity to capture fine-grained temporal dependencies. Our empirical evaluations show that FVDM outperforms state-of-the-art methods in video generation quality, while also excelling in extended tasks.
  arXiv  Detail & Related papers  (2024-10-04T05:47:39Z)
• LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Models [133.088893990272]
  We learn a high-quality text-to-video (T2V) generative model by leveraging a pre-trained text-to-image (T2I) model as a basis. We propose LaVie, an integrated video generation framework that operates on cascaded video latent diffusion models.
  arXiv  Detail & Related papers  (2023-09-26T17:52:03Z)
• VideoGen: A Reference-Guided Latent Diffusion Approach for High Definition Text-to-Video Generation [73.54366331493007]
  VideoGen is a text-to-video generation approach, which can generate a high-definition video with high frame fidelity and strong temporal consistency. We leverage an off-the-shelf text-to-image generation model, e.g., Stable Diffusion, to generate an image with high content quality from the text prompt.
  arXiv  Detail & Related papers  (2023-09-01T11:14:43Z)
• Edit Temporal-Consistent Videos with Image Diffusion Model [49.88186997567138]
  Large-scale text-to-image (T2I) diffusion models have been extended for text-guided video editing. T achieves state-of-the-art performance in both video temporal consistency and video editing capability.
  arXiv  Detail & Related papers  (2023-08-17T16:40:55Z)
• Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation [93.18163456287164]
  This paper proposes a novel text-guided video-to-video translation framework to adapt image models to videos. Our framework achieves global style and local texture temporal consistency at a low cost.
  arXiv  Detail & Related papers  (2023-06-13T17:52:23Z)
• Task Agnostic Restoration of Natural Video Dynamics [10.078712109708592]
  In many video restoration/translation tasks, image processing operations are na"ively extended to the video domain by processing each frame independently. We propose a general framework for this task that learns to infer and utilize consistent motion dynamics from inconsistent videos to mitigate the temporal flicker. The proposed framework produces SOTA results on two benchmark datasets, DAVIS and videvo.net, processed by numerous image processing applications.
  arXiv  Detail & Related papers  (2022-06-08T09:00:31Z)
• Capturing Video Frame Rate Variations via Entropic Differencing [63.749184706461826]
  We propose a novel statistical entropic differencing method based on a Generalized Gaussian Distribution model. Our proposed model correlates very well with subjective scores in the recently proposed LIVE-YT-HFR database.
  arXiv  Detail & Related papers  (2020-06-19T22:16:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.