LongDiff: Training-Free Long Video Generation in One Go

• URL: http://arxiv.org/abs/2503.18150v1
• Date: Sun, 23 Mar 2025 17:34:57 GMT
• Title: LongDiff: Training-Free Long Video Generation in One Go
• Authors: Zhuoling Li, Hossein Rahmani, Qiuhong Ke, Jun Liu,
• Abstract summary: LongDiff is a training-free method consisting of Position Mapping (PM) and Informative Frame Selection (IFS)<n>Our method tackles two key challenges that hinder short-to-long video generation generalization: temporal position ambiguity and information dilution.<n>Our method unlocks the potential of off-the-shelf video diffusion models to achieve high-quality long video generation in one go.
• Score: 27.38597403230757
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video diffusion models have recently achieved remarkable results in video generation. Despite their encouraging performance, most of these models are mainly designed and trained for short video generation, leading to challenges in maintaining temporal consistency and visual details in long video generation. In this paper, we propose LongDiff, a novel training-free method consisting of carefully designed components \ -- Position Mapping (PM) and Informative Frame Selection (IFS) \ -- to tackle two key challenges that hinder short-to-long video generation generalization: temporal position ambiguity and information dilution. Our LongDiff unlocks the potential of off-the-shelf video diffusion models to achieve high-quality long video generation in one go. Extensive experiments demonstrate the efficacy of our method.

Related papers

• VideoMerge: Towards Training-free Long Video Generation [46.108622251662176]
  Long video generation remains a challenging and compelling topic in computer vision.<n>We propose VideoMerge, a training-free method that can be seamlessly adapted to merge short videos.
  arXiv  Detail & Related papers  (2025-03-13T00:47:59Z)
• LVD-2M: A Long-take Video Dataset with Temporally Dense Captions [68.88624389174026]
  We introduce a new pipeline for selecting high-quality long-take videos and generating temporally dense captions. Specifically, we define a set of metrics to quantitatively assess video quality including scene cuts, dynamic degrees, and semantic-level quality. We curate the first long-take video dataset, LVD-2M, comprising 2 million long-take videos, each covering more than 10 seconds and annotated with temporally dense captions.
  arXiv  Detail & Related papers  (2024-10-14T17:59:56Z)
• FreeLong: Training-Free Long Video Generation with SpectralBlend Temporal Attention [57.651429116402554]
  This paper investigates a straightforward and training-free approach to extend an existing short video diffusion model for consistent long video generation. We find that directly applying the short video diffusion model to generate long videos can lead to severe video quality degradation. Motivated by this, we propose a novel solution named FreeLong to balance the frequency distribution of long video features during the denoising process.
  arXiv  Detail & Related papers  (2024-07-29T11:52:07Z)
• Anchored Diffusion for Video Face Reenactment [17.343307538702238]
  We introduce Anchored Diffusion, a novel method for synthesizing relatively long and seamless videos. We train our model on video sequences with random non-uniform temporal spacing and incorporate temporal information via external guidance. During inference, we leverage the transformer architecture to modify the diffusion process, generating a batch of non-uniform sequences anchored to a common frame.
  arXiv  Detail & Related papers  (2024-07-21T13:14:17Z)
• Video Generation Beyond a Single Clip [76.5306434379088]
  Video generation models can only generate video clips that are relatively short compared with the length of real videos. To generate long videos covering diverse content and multiple events, we propose to use additional guidance to control the video generation process. The proposed approach is complementary to existing efforts on video generation, which focus on generating realistic video within a fixed time window.
  arXiv  Detail & Related papers  (2023-04-15T06:17:30Z)
• Latent Video Diffusion Models for High-Fidelity Long Video Generation [58.346702410885236]
  We introduce lightweight video diffusion models using a low-dimensional 3D latent space. We also propose hierarchical diffusion in the latent space such that longer videos with more than one thousand frames can be produced. Our framework generates more realistic and longer videos than previous strong baselines.
  arXiv  Detail & Related papers  (2022-11-23T18:58:39Z)
• Generating Long Videos of Dynamic Scenes [66.56925105992472]
  We present a video generation model that reproduces object motion, changes in camera viewpoint, and new content that arises over time. A common failure case is for content to never change due to over-reliance on inductive biases to provide temporal consistency.
  arXiv  Detail & Related papers  (2022-06-07T16:29:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.