InfLVG: Reinforce Inference-Time Consistent Long Video Generation with GRPO

• URL: http://arxiv.org/abs/2505.17574v1
• Date: Fri, 23 May 2025 07:33:25 GMT
• Title: InfLVG: Reinforce Inference-Time Consistent Long Video Generation with GRPO
• Authors: Xueji Fang, Liyuan Ma, Zhiyang Chen, Mingyuan Zhou, Guo-jun Qi,
• Abstract summary: InfLVG is an inference-time framework that enables coherent long video generation without requiring additional long-form video data.<n>We show that InfLVG can extend video length by up to 9$times$, achieving strong consistency and semantic fidelity across scenes.
• Score: 73.33751812982342
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in text-to-video generation, particularly with autoregressive models, have enabled the synthesis of high-quality videos depicting individual scenes. However, extending these models to generate long, cross-scene videos remains a significant challenge. As the context length grows during autoregressive decoding, computational costs rise sharply, and the model's ability to maintain consistency and adhere to evolving textual prompts deteriorates. We introduce InfLVG, an inference-time framework that enables coherent long video generation without requiring additional long-form video data. InfLVG leverages a learnable context selection policy, optimized via Group Relative Policy Optimization (GRPO), to dynamically identify and retain the most semantically relevant context throughout the generation process. Instead of accumulating the entire generation history, the policy ranks and selects the top-$K$ most contextually relevant tokens, allowing the model to maintain a fixed computational budget while preserving content consistency and prompt alignment. To optimize the policy, we design a hybrid reward function that jointly captures semantic alignment, cross-scene consistency, and artifact reduction. To benchmark performance, we introduce the Cross-scene Video Benchmark (CsVBench) along with an Event Prompt Set (EPS) that simulates complex multi-scene transitions involving shared subjects and varied actions/backgrounds. Experimental results show that InfLVG can extend video length by up to 9$\times$, achieving strong consistency and semantic fidelity across scenes. Our code is available at https://github.com/MAPLE-AIGC/InfLVG.

Related papers

• LoViC: Efficient Long Video Generation with Context Compression [68.22069741704158]
  We introduce LoViC, a DiT-based framework trained on million-scale open-domain videos.<n>At the core of our approach is FlexFormer, an expressive autoencoder that jointly compresses video and text into unified latent representations.
  arXiv  Detail & Related papers  (2025-07-17T09:46:43Z)
• AdaVideoRAG: Omni-Contextual Adaptive Retrieval-Augmented Efficient Long Video Understanding [73.60257070465377]
  AdaVideoRAG is a novel framework that adapts retrieval based on query complexity using a lightweight intent classifier.<n>Our framework employs an Omni-Knowledge Indexing module to build hierarchical databases from text (captions, ASR, OCR), visual features, and semantic graphs.<n> Experiments demonstrate improved efficiency and accuracy for long-video understanding, with seamless integration into existing MLLMs.
  arXiv  Detail & Related papers  (2025-06-16T15:18:15Z)
• SceneRAG: Scene-level Retrieval-Augmented Generation for Video Understanding [6.980340270823506]
  We present SceneRAG, a framework to segment videos into narrative-consistent scenes.<n>For each scene, the framework fuses information from both visual and textual modalities to extract entity relations.<n>Experiments on the LongerVideos benchmark, featuring over 134 hours of diverse content, confirm that SceneRAG substantially outperforms prior baselines.
  arXiv  Detail & Related papers  (2025-06-09T10:00:54Z)
• HiTVideo: Hierarchical Tokenizers for Enhancing Text-to-Video Generation with Autoregressive Large Language Models [63.65066762436074]
  HiTVideo aims to address the potential limitations of existing video tokenizers in text-to-video generation tasks.<n>It utilizes a 3D causal VAE with a multi-layer discrete token framework, encoding video content into hierarchically structured codebooks.
  arXiv  Detail & Related papers  (2025-03-14T15:36:39Z)
• Video Diffusion Transformers are In-Context Learners [31.736838809714726]
  This paper investigates a solution for enabling in-context capabilities of video diffusion transformers.<n>We propose a simple pipeline to leverage in-context generation: ($textbfii$) videos along spacial or time dimension.<n>Our framework presents a valuable tool for the research community and offers critical insights for advancing product-level controllable video generation systems.
  arXiv  Detail & Related papers  (2024-12-14T10:39:55Z)
• SALOVA: Segment-Augmented Long Video Assistant for Targeted Retrieval and Routing in Long-Form Video Analysis [52.050036778325094]
  We introduce SALOVA: Segment-Augmented Video Assistant, a novel video-LLM framework designed to enhance the comprehension of lengthy video content.<n>We present a high-quality collection of 87.8K long videos, each densely captioned at the segment level to enable models to capture scene continuity and maintain rich context.<n>Our framework mitigates the limitations of current video-LMMs by allowing for precise identification and retrieval of relevant video segments in response to queries.
  arXiv  Detail & Related papers  (2024-11-25T08:04:47Z)
• Generating Long Videos of Dynamic Scenes [66.56925105992472]
  We present a video generation model that reproduces object motion, changes in camera viewpoint, and new content that arises over time. A common failure case is for content to never change due to over-reliance on inductive biases to provide temporal consistency.
  arXiv  Detail & Related papers  (2022-06-07T16:29:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.