Evaluating Design Video Generation: Metrics for Compositional Fidelity

• URL: http://arxiv.org/abs/2605.16223v1
• Date: Fri, 15 May 2026 17:34:05 GMT
• Title: Evaluating Design Video Generation: Metrics for Compositional Fidelity
• Abstract summary: This paper provides a fully automated evaluation framework organized across four dimensions: layout fidelity, motion correctness, temporal quality, and content fidelity.<n>This eliminates the reliance on subjective human evaluation and establishes a common basis for benchmarking progress in the field.
• Score: 6.297657588574816
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generative video models are increasingly used in design animation tasks, yet no standardized evaluation framework exists for this domain. Unlike natural video generation, design animation imposes structured constraints: specific components shall animate with prescribed motion types, directions, speed and timing, while non-animated regions must remain stable and layout structure must be preserved. This paper provides a fully automated evaluation framework organized across four dimensions: layout fidelity, motion correctness, temporal quality, and content fidelity. This eliminates the reliance on subjective human evaluation and establishes a common basis for benchmarking progress in the field.

Related papers

• VideoCoCo: Code-as-CoT for Physically-Consistent Video Generation via an Agentic Dual-Engine System [77.32035315337522]
  Text-video models struggle to generate physically consistent dynamics the temporal evolution of a scene must be inferred implicitly from a compressed text prompt.<n>Existing chain-of-thought approaches introduce intermediate plans or visual states, but these representations are typically non-cutable or temporally sparse.<n>We introduce VideoCo, an agentic dual-engine framework in which executable Blender code serves as a process-level chain of thought.
  arXiv  Detail & Related papers  (2026-07-29T18:38:23Z)
• Compositional Video Generation via Inference-Time Guidance [69.53614395025632]
  Text-to-video diffusion models often fail on prompts requiring compositional understanding.<n>We propose textbfCVG, an inference-time guidance method for improving compositional faithfulness in frozen text-to-video models.
  arXiv  Detail & Related papers  (2026-05-14T15:50:25Z)
• AnimationBench: Are Video Models Good at Character-Centric Animation? [67.15372653996833]
  AnimationBench is the first systematic benchmark for evaluating animation image-to-video generation.<n>It operationalizes the Twelve Basic Principles of Animation and IP Preservation into measurable evaluation dimensions.<n>It exposes animation-specific quality differences overlooked by realism-oriented benchmarks.
  arXiv  Detail & Related papers  (2026-04-16T17:57:08Z)
• Graphic-Design-Bench: A Comprehensive Benchmark for Evaluating AI on Graphic Design Tasks [7.841779848822317]
  GraphicDesignBench (GDB) is the first comprehensive benchmark suite designed specifically to evaluate AI models on the full breadth of professional graphic design tasks.<n>The suite comprises 50 tasks organized along five axes: layout, typography, infographics, template & design semantics and animation.<n>We evaluate a set of frontier closed-source models using a standardized metric taxonomy covering spatial accuracy, perceptual quality, text fidelity, semantic alignment, and structural validity.
  arXiv  Detail & Related papers  (2026-04-05T17:20:08Z)
• SAMA: Factorized Semantic Anchoring and Motion Alignment for Instruction-Guided Video Editing [76.349958946335]
  We present SAMA (factorized Semantic Anchoring and Motion Alignment), a framework that factorizes video editing into semantic anchoring and motion modeling.<n>First, we introduce Semantic Anchoring, which establishes a reliable visual anchor by jointly predicting semantic tokens and video latents at sparse anchor frames.<n>Second, Motion Alignment pre-trains the same backbone on motion-centric video restoration pretext tasks.
  arXiv  Detail & Related papers  (2026-03-19T17:59:51Z)
• Screen, Match, and Cache: A Training-Free Causality-Consistent Reference Frame Framework for Human Animation [44.20260674331104]
  FrameCache is a training-free three-stage framework consisting of Screen, Cache, and Match.<n>Experiments on standard benchmarks demonstrate that FrameCache consistently improves temporal coherence and visual stability.
  arXiv  Detail & Related papers  (2025-12-13T08:45:03Z)
• Frame In-N-Out: Unbounded Controllable Image-to-Video Generation [23.231729778050873]
  We focus on a commonly used yet underexplored cinematic technique known as Frame In and Frame Out.<n>We introduce a new dataset that is curated semi-automatically, an efficient identity-preserving motion-controllable video Diffusion Transformer architecture, and a comprehensive evaluation protocol targeting this task.
  arXiv  Detail & Related papers  (2025-05-27T17:56:07Z)
• Tracktention: Leveraging Point Tracking to Attend Videos Faster and Better [61.381599921020175]
  Temporal consistency is critical in video prediction to ensure that outputs are coherent and free of artifacts.<n>Traditional methods, such as temporal attention and 3D convolution, may struggle with significant object motion.<n>We propose the Tracktention Layer, a novel architectural component that explicitly integrates motion information using point tracks.
  arXiv  Detail & Related papers  (2025-03-25T17:58:48Z)
• TC-Bench: Benchmarking Temporal Compositionality in Text-to-Video and Image-to-Video Generation [97.96178992465511]
  We argue that generated videos should incorporate the emergence of new concepts and their relation transitions like in real-world videos as time progresses. To assess the Temporal Compositionality of video generation models, we propose TC-Bench, a benchmark of meticulously crafted text prompts, corresponding ground truth videos, and robust evaluation metrics.
  arXiv  Detail & Related papers  (2024-06-12T21:41:32Z)
• Unsupervised Coherent Video Cartoonization with Perceptual Motion Consistency [89.75731026852338]
  We propose a spatially-adaptive alignment framework with perceptual motion consistency for coherent video cartoonization. We devise the semantic correlative map as a style-independent, global-aware regularization on the perceptual consistency motion. Our method is able to generate highly stylistic and temporal consistent cartoon videos.
  arXiv  Detail & Related papers  (2022-04-02T07:59:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.