NarrLV: Towards a Comprehensive Narrative-Centric Evaluation for Long Video Generation Models

• URL: http://arxiv.org/abs/2507.11245v2
• Date: Tue, 29 Jul 2025 17:22:40 GMT
• Title: NarrLV: Towards a Comprehensive Narrative-Centric Evaluation for Long Video Generation Models
• Authors: X. Feng, H. Yu, M. Wu, S. Hu, J. Chen, C. Zhu, J. Wu, X. Chu, K. Huang,
• Abstract summary: The goal of long video generation tasks is not only to extend video duration but also to accurately express richer narrative content within longer videos.<n>Our proposed NarrLV is the first benchmark to comprehensively evaluate the Narrative expression capabilities of Long Video generation models.
• Score: 0.17413461132662073
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With the rapid development of foundation video generation technologies, long video generation models have exhibited promising research potential thanks to expanded content creation space. Recent studies reveal that the goal of long video generation tasks is not only to extend video duration but also to accurately express richer narrative content within longer videos. However, due to the lack of evaluation benchmarks specifically designed for long video generation models, the current assessment of these models primarily relies on benchmarks with simple narrative prompts (e.g., VBench). To the best of our knowledge, our proposed NarrLV is the first benchmark to comprehensively evaluate the Narrative expression capabilities of Long Video generation models. Inspired by film narrative theory, (i) we first introduce the basic narrative unit maintaining continuous visual presentation in videos as Temporal Narrative Atom (TNA), and use its count to quantitatively measure narrative richness. Guided by three key film narrative elements influencing TNA changes, we construct an automatic prompt generation pipeline capable of producing evaluation prompts with a flexibly expandable number of TNAs. (ii) Then, based on the three progressive levels of narrative content expression, we design an effective evaluation metric using the MLLM-based question generation and answering framework. (iii) Finally, we conduct extensive evaluations on existing long video generation models and the foundation generation models. Experimental results demonstrate that our metric aligns closely with human judgments. The derived evaluation outcomes reveal the detailed capability boundaries of current video generation models in narrative content expression.

Related papers

• VideoAuteur: Towards Long Narrative Video Generation [22.915448471769384]
  We present a large-scale cooking video dataset designed to advance long-form narrative generation in the cooking domain.<n>We introduce a Long Narrative Video Director to enhance both visual and semantic coherence in generated videos.<n>Our method demonstrates substantial improvements in generating visually detailed and semantically aligneds.
  arXiv  Detail & Related papers  (2025-01-10T18:52:11Z)
• Is Your World Simulator a Good Story Presenter? A Consecutive Events-Based Benchmark for Future Long Video Generation [71.32108638269517]
  We introduce StoryEval, a story-oriented benchmark to assess text-to-video (T2V) models' story-completion capabilities.<n>StoryEval features 423 prompts spanning 7 classes, each representing short stories composed of 2-4 consecutive events.<n>We employ advanced vision-language models, such as GPT-4V and LLaVA-OV-Chat-72B, to verify the completion of each event in the generated videos.
  arXiv  Detail & Related papers  (2024-12-17T23:00:42Z)
• TeaserGen: Generating Teasers for Long Documentaries [59.8220642722399]
  We present DocumentaryNet, a collection of 1,269 documentaries paired with their teasers. We propose a new two-stage system for generating teasers from long documentaries.
  arXiv  Detail & Related papers  (2024-10-08T01:00:09Z)
• MovieDreamer: Hierarchical Generation for Coherent Long Visual Sequence [62.72540590546812]
  MovieDreamer is a novel hierarchical framework that integrates the strengths of autoregressive models with diffusion-based rendering. We present experiments across various movie genres, demonstrating that our approach achieves superior visual and narrative quality.
  arXiv  Detail & Related papers  (2024-07-23T17:17:05Z)
• Scaling Up Video Summarization Pretraining with Large Language Models [73.74662411006426]
  We introduce an automated and scalable pipeline for generating a large-scale video summarization dataset. We analyze the limitations of existing approaches and propose a new video summarization model that effectively addresses them. Our work also presents a new benchmark dataset that contains 1200 long videos each with high-quality summaries annotated by professionals.
  arXiv  Detail & Related papers  (2024-04-04T11:59:06Z)
• STREAM: Spatio-TempoRal Evaluation and Analysis Metric for Video Generative Models [6.855409699832414]
  Video generative models struggle to generate even short video clips. Current video evaluation metrics are simple adaptations of image metrics by switching the embeddings with video embedding networks. We propose STREAM, a new video evaluation metric uniquely designed to independently evaluate spatial and temporal aspects.
  arXiv  Detail & Related papers  (2024-01-30T08:18:20Z)
• FreeNoise: Tuning-Free Longer Video Diffusion via Noise Rescheduling [85.60543452539076]
  Existing video generation models are typically trained on a limited number of frames, resulting in the inability to generate high-fidelity long videos during inference. This study explores the potential of extending the text-driven capability to generate longer videos conditioned on multiple texts. We propose FreeNoise, a tuning-free and time-efficient paradigm to enhance the generative capabilities of pretrained video diffusion models.
  arXiv  Detail & Related papers  (2023-10-23T17:59:58Z)
• EvalCrafter: Benchmarking and Evaluating Large Video Generation Models [70.19437817951673]
  We argue that it is hard to judge the large conditional generative models from the simple metrics since these models are often trained on very large datasets with multi-aspect abilities. Our approach involves generating a diverse and comprehensive list of 700 prompts for text-to-video generation. Then, we evaluate the state-of-the-art video generative models on our carefully designed benchmark, in terms of visual qualities, content qualities, motion qualities, and text-video alignment with 17 well-selected objective metrics.
  arXiv  Detail & Related papers  (2023-10-17T17:50:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.