KeyFrame-Compass: Towards Comprehensive Evaluation of Keyframe-Conditioned Video Generation

• URL: http://arxiv.org/abs/2607.14202v1
• Date: Wed, 15 Jul 2026 17:46:12 GMT
• Title: KeyFrame-Compass: Towards Comprehensive Evaluation of Keyframe-Conditioned Video Generation
• Abstract summary: Video generation increasingly relies on granular synthesis-based synthesis, where creators specify a sequence of reference images to guide generation.<n>Although recent models support multi-keyframe conditioning, it remains unclear whether they can faithfully reproduce the prescribeds while maintaining overall video quality.<n>We present KeyFrame, the first comprehensive benchmark for evaluating granular synthesis-based video generation.
• Score: 25.29345016590953
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Video generation increasingly relies on keyframe-based workflows, where creators specify a sequence of reference images to guide generation. Although recent models support multi-keyframe conditioning, it remains unclear whether they can faithfully reproduce the prescribed keyframes while maintaining overall video quality. We present KeyFrame-Compass, the first comprehensive benchmark for evaluating keyframe-conditioned video generation. The benchmark contains 386 carefully curated samples spanning three application domains, two video structures, two prompt granularities, two conditioning formats, and four keyframe densities, enabling controlled analysis under diverse generation settings. We further introduce an automated evaluation framework that jointly measures keyframe execution and overall video quality. Specifically, we decompose keyframe execution into six complementary metrics covering presence, fidelity, temporal ordering, localization, persistence, and uniqueness, while assessing overall video quality through evidence-grounded MLLM judgments augmented with specialized perception models. Experiments on nine representative video generation systems reveal several fundamental limitations. Current models exhibit a clear trade-off between faithful keyframe execution and natural video synthesis. Their performance further degrades as keyframe constraints become denser and most open-source models also fail to interpret storyboard-grid inputs as temporally ordered keyframe sequences.

Related papers

• Controllable Generative Video Compression [11.376749911846302]
  We propose Controllable Generative Video Compression (CGVC) paradigm to faithfully generate details guided by multiple visual conditions.<n>CGVC outperforms previous perceptual video compression method in terms of both signal fidelity and perceptual quality.
  arXiv  Detail & Related papers  (2026-04-08T04:11:27Z)
• AlcheMinT: Fine-grained Temporal Control for Multi-Reference Consistent Video Generation [58.844504598618094]
  We propose AlcheMinT, a unified framework that introduces explicit timestamps conditioning for subject-driven video generation.<n>Our approach introduces a novel positional encoding mechanism that unlocks the encoding of temporal intervals, associated in our case with subject identities.<n>We incorporate subject-descriptive text tokens to strengthen binding between visual identity and video captions, mitigating ambiguity during generation.
  arXiv  Detail & Related papers  (2025-12-11T18:59:34Z)
• K-frames: Scene-Driven Any-k Keyframe Selection for long video understanding [38.06179287702453]
  K-frames is a novel paradigm for scene-driven selection that preserves temporal continuity.<n>Instead of selecting individual frames, K-frames predicts semantically coherent, query-relevant clips.<n>K-frames provides an effective, interpretable, and plug-and-play solution for selection at various scales.
  arXiv  Detail & Related papers  (2025-10-14T06:23:22Z)
• From Captions to Keyframes: KeyScore for Multimodal Frame Scoring and Video-Language Understanding [1.3856027745141806]
  KeyScore is a caption-aware frame scoring method that combines semantic similarity to captions, temporal representativeness, and contextual drop impact.<n>Our method enhances both efficiency and performance, enabling scalable and caption-grounded video understanding.
  arXiv  Detail & Related papers  (2025-10-07T23:02:27Z)
• Threading Keyframe with Narratives: MLLMs as Strong Long Video Comprehenders [62.58375366359421]
  Multimodal Large Language Models (MLLMs) for long video understanding remains a challenging problem.<n>Traditional uniform sampling leads to selection of irrelevant content.<n>Post-training MLLMs on thousands of frames imposes a substantial computational burden.<n>We propose threadings with narratives (Nar-KFC) to facilitate effective and efficient long video perception.
  arXiv  Detail & Related papers  (2025-05-30T03:04:28Z)
• Scaling Video-Language Models to 10K Frames via Hierarchical Differential Distillation [20.67434288227437]
  ViLAMP is a hierarchical video-language model that processes hour-long videos at "mixed precision"<n>ViLAMP retains full information ins while reducing non-keyframes to their most salient features, resembling mixed-precision training.<n> Notably, ViLAMP can process ultra-long videos (up to 10K frames) on a single NVIDIA A100 GPU.
  arXiv  Detail & Related papers  (2025-04-03T09:55:09Z)
• Adaptive Keyframe Sampling for Long Video Understanding [75.7837692594814]
  This paper presents a simple yet effective algorithm named Adaptive Keyframe Sampling (AKS)<n>It inserts a plug-and-play module known as Adaptive Keyframe Sampling (AKS) which aims to maximize the useful information with a fixed number of video tokens.<n>Experiments on two long video understanding benchmarks validate that AKS improves video QA accuracy upon selecting informative encounters.
  arXiv  Detail & Related papers  (2025-02-28T17:46:29Z)
• The Devil is in Temporal Token: High Quality Video Reasoning Segmentation [68.33080352141653]
  Methods for Video Reasoning rely heavily on a single special token to represent the object in the video.<n>We propose VRS-HQ, an end-to-end video reasoning segmentation approach.<n>Our results highlight the strong temporal reasoning and segmentation capabilities of our method.
  arXiv  Detail & Related papers  (2025-01-15T03:17:24Z)
• M3-CVC: Controllable Video Compression with Multimodal Generative Models [17.49397141459785]
  M3-CVC is a controllable video compression framework incorporating generative models.<n>We show that M3-CVC significantly outperforms the state-the-art VVC standard in ultralow scenarios.
  arXiv  Detail & Related papers  (2024-11-24T11:56:59Z)
• Let's Think Frame by Frame with VIP: A Video Infilling and Prediction Dataset for Evaluating Video Chain-of-Thought [62.619076257298204]
  We motivate framing video reasoning as the sequential understanding of a small number of video reasonings. We introduce VIP, an inference-time challenge dataset designed to explore models' reasoning capabilities through video chain-of-thought. We benchmark GPT-4, GPT-3, and VICUNA on VIP, demonstrate the performance gap in complex video reasoning tasks, and encourage future work.
  arXiv  Detail & Related papers  (2023-05-23T10:26:42Z)
• MHSCNet: A Multimodal Hierarchical Shot-aware Convolutional Network for Video Summarization [61.69587867308656]
  We propose a multimodal hierarchical shot-aware convolutional network, denoted as MHSCNet, to enhance the frame-wise representation. Based on the learned shot-aware representations, MHSCNet can predict the frame-level importance score in the local and global view of the video.
  arXiv  Detail & Related papers  (2022-04-18T14:53:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.