Long-CODE: Isolating Pure Long-Context as an Orthogonal Dimension in Video Evaluation

• URL: http://arxiv.org/abs/2604.17428v1
• Date: Sun, 19 Apr 2026 13:17:34 GMT
• Title: Long-CODE: Isolating Pure Long-Context as an Orthogonal Dimension in Video Evaluation
• Abstract summary: We argue that long-video metrics should be disentangled from short-video assessments.<n>We introduce a suite of long-video attribute corruption tests and a novel long-video metric based on shot dynamics.<n>Our proposed metrics achieve state-of-the-art correlation with human judgments.
• Score: 16.64717198652712
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As video generation models achieve unprecedented capabilities, the demand for robust video evaluation metrics becomes increasingly critical. Traditional metrics are intrinsically tailored for short-video evaluation, predominantly assessing frame-level visual quality and localized temporal smoothness. However, as state-of-the-art video generation models scale to generate longer videos, these metrics fail to capture essential long-range characteristics, such as narrative richness and global causal consistency. Recognizing that short-term visual perception and long-context attributes are fundamentally orthogonal dimensions, we argue that long-video metrics should be disentangled from short-video assessments. In this paper, we focus on the rigorous justification and design of a dedicated framework for long-video evaluation. We first introduce a suite of long-video attribute corruption tests, exposing the critical limitations of existing hort-video metrics from their insensitivity to structural inconsistencies, such as shot-level perturbations and narrative shuffling. To bridge this gap, we design a novel long-video metric based on shot dynamics, which is highly sensitive to the long-range testing framework. Furthermore, we introduce Long-CODE (Long-Context as an Orthogonal Dimension for video Evaluation), a specialized dataset designed to benchmark long-video evaluation, with human annotations isolated specifically to genuine long-range characteristics. Extensive experiments show that our proposed metrics achieve state-of-the-art correlation with human judgments. Ultimately, our metric and benchmark seamlessly complement existing short-video standards, establishing a holistic and unbiased evaluation paradigm for video generation models.

Related papers

• MBench: A Comprehensive Benchmark on Memory Capability for Video World Models [36.71271805993198]
  We present textbfMBench, a benchmark dedicated to quantifying and evaluating the memory capability of video world models.<n>Our benchmark is built upon rigorously curated real-captured long videos, and evaluated by rule-based quantitative matrices and VLM.
  arXiv  Detail & Related papers  (2026-05-30T16:17:33Z)
• LongVPO: From Anchored Cues to Self-Reasoning for Long-Form Video Preference Optimization [20.692871849527815]
  LongVPO is a framework that enables vision-context models to robustly understand ultra-long videos without any long-video annotations.<n>With only 16K synthetic examples and no costly human labels, LongVPO outperforms state-of-the-art open-source models on multiple long-video benchmarks.
  arXiv  Detail & Related papers  (2026-02-02T17:03:37Z)
• HumanVideo-MME: Benchmarking MLLMs for Human-Centric Video Understanding [120.84817886550765]
  Multimodal Large Language Models (MLLMs) have demonstrated significant advances in visual understanding tasks involving both images and videos.<n>Existing human-centric benchmarks predominantly emphasize video generation quality and action recognition, while overlooking essential perceptual and cognitive abilities required in human-centered scenarios.<n>We propose a rigorously curated benchmark designed to provide a more holistic evaluation of MLLMs in human-centric video understanding.
  arXiv  Detail & Related papers  (2025-07-07T11:52:24Z)
• Long-RVOS: A Comprehensive Benchmark for Long-term Referring Video Object Segmentation [51.2732688481343]
  We introduce Longtextbf-RVOS, a large-scale benchmark for long-term referring object segmentation.<n>Long-RVOS contains 2,000+ videos of an average duration exceeding 60 seconds, covering a variety of objects.<n>Unlike previous benchmarks that rely solely on the per-frame spatial evaluation, we introduce two metrics to assess the temporal andtemporal consistency.
  arXiv  Detail & Related papers  (2025-05-19T04:52:31Z)
• SALOVA: Segment-Augmented Long Video Assistant for Targeted Retrieval and Routing in Long-Form Video Analysis [52.050036778325094]
  We introduce SALOVA: Segment-Augmented Video Assistant, a novel video-LLM framework designed to enhance the comprehension of lengthy video content.<n>We present a high-quality collection of 87.8K long videos, each densely captioned at the segment level to enable models to capture scene continuity and maintain rich context.<n>Our framework mitigates the limitations of current video-LMMs by allowing for precise identification and retrieval of relevant video segments in response to queries.
  arXiv  Detail & Related papers  (2024-11-25T08:04:47Z)
• STREAM: Spatio-TempoRal Evaluation and Analysis Metric for Video Generative Models [6.855409699832414]
  Video generative models struggle to generate even short video clips. Current video evaluation metrics are simple adaptations of image metrics by switching the embeddings with video embedding networks. We propose STREAM, a new video evaluation metric uniquely designed to independently evaluate spatial and temporal aspects.
  arXiv  Detail & Related papers  (2024-01-30T08:18:20Z)
• EvalCrafter: Benchmarking and Evaluating Large Video Generation Models [70.19437817951673]
  We argue that it is hard to judge the large conditional generative models from the simple metrics since these models are often trained on very large datasets with multi-aspect abilities. Our approach involves generating a diverse and comprehensive list of 700 prompts for text-to-video generation. Then, we evaluate the state-of-the-art video generative models on our carefully designed benchmark, in terms of visual qualities, content qualities, motion qualities, and text-video alignment with 17 well-selected objective metrics.
  arXiv  Detail & Related papers  (2023-10-17T17:50:46Z)
• Revisiting Kernel Temporal Segmentation as an Adaptive Tokenizer for Long-form Video Understanding [57.917616284917756]
  Real-world videos are often several minutes long with semantically consistent segments of variable length. A common approach to process long videos is applying a short-form video model over uniformly sampled clips of fixed temporal length. This approach neglects the underlying nature of long videos since fixed-length clips are often redundant or uninformative.
  arXiv  Detail & Related papers  (2023-09-20T18:13:32Z)
• Generating Long Videos of Dynamic Scenes [66.56925105992472]
  We present a video generation model that reproduces object motion, changes in camera viewpoint, and new content that arises over time. A common failure case is for content to never change due to over-reliance on inductive biases to provide temporal consistency.
  arXiv  Detail & Related papers  (2022-06-07T16:29:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.