CLIP-CC-Bench: Evaluating Paragraph-Level Video Descriptions in Video-Language Models

• URL: http://arxiv.org/abs/2608.04302v1
• Date: Wed, 05 Aug 2026 00:20:23 GMT
• Title: CLIP-CC-Bench: Evaluating Paragraph-Level Video Descriptions in Video-Language Models
• Abstract summary: We introduce CLIP-CC-Bench, an evaluation suite for long-form video description built from 5 hours of movie content.<n>The evaluation suite employs an ensemble of five state-of-the-art embedding models to increase reliability and single-model bias.<n>Using this framework, we evaluate 17 state-of-the-art video-language models and report both their Borda-aggregated rankings and their average scores on CLIP-CC-Bench.
• Score: 3.1498833540989413
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Benchmarking video-language models has largely focused on short clips and single-sentence metrics, leaving open whether current systems can generate accurate long-form, paragraph-level descriptions. We introduce CLIP-CC-Bench, an evaluation suite for long-form video description built from 5 hours of movie content segmented into 90-second clips, each paired with an expert-written paragraph-style reference. The evaluation suite employs an ensemble of five state-of-the-art LLM-based embedding models to increase reliability and mitigate single-model bias, and applies two complementary methodologies: (i) coarse-grained semantic matching and (ii) fine-grained semantic matching to compare model-generated descriptions against CLIP-CC-Bench references. Using this framework, we evaluate 17 state-of-the-art video-language models and report both their Borda-aggregated rankings and their average scores on CLIP-CC-Bench. We further quantify the protocol's internal reliability through inter-judge agreement and bootstrap ranking stability. We release standardized evaluation scripts, model outputs, and aggregation tools at https://github.com/Multimodal-Intelligence-Lab/CLIP-CC-Bench to support reproducibility. CLIP-CC-Bench provides a practical evaluation framework for long-form video description, filling a gap left by existing short-clip and QA-only benchmarks.

Related papers

• VISTA: Dense Multi-Label Classroom Coding with Vision-Language Models [0.0]
  We argue that multimodal benchmarking benefits from methods taken from research communities that have already invested in strategies to ensure reliability.<n>We recast COPUS as a video benchmark for multimodal foundation models, where it provides a dense set of structured labels.<n>We propose VISTA, a baseline that runs MiniCPM-V-4.5 over a dense sliding window, refines its per-window outputs with a lightweight multi-layer perceptron (MLP) head trained on top of the frozen backbone.
  arXiv  Detail & Related papers  (2026-09-03T23:13:56Z)
• Reasoning as Intersection: Consensus-Frame Alignment for Visual Focus in Video-MLLMs [81.04673240949074]
  Reinforcement learning has improved the reasoning ability of large language models.<n>Applying outcome-only rewards to video multimodal large language models provides limited guidance on which visual evidence should support the answer.<n>We introduce Consensus Frame GRPO, a temporal-annotation-free process-level reward framework for evidence-aware video reasoning.
  arXiv  Detail & Related papers  (2026-06-16T19:42:54Z)
• UniEditBench: A Unified and Cost-Effective Benchmark for Image and Video Editing via Distilled MLLMs [19.634771714749544]
  We present UniEditBench, a unified benchmark for image and video editing.<n>UniEditBench supports reconstruction-based and instruction-driven methods under a shared protocol.<n>We distill a high-capacity MLLM judge into lightweight 4B/8B evaluators that provide multi-dimensional scoring over structural fidelity, text alignment, background consistency, naturalness, and temporal-spatial consistency for videos.
  arXiv  Detail & Related papers  (2026-04-17T09:21:48Z)
• LongVPO: From Anchored Cues to Self-Reasoning for Long-Form Video Preference Optimization [20.692871849527815]
  LongVPO is a framework that enables vision-context models to robustly understand ultra-long videos without any long-video annotations.<n>With only 16K synthetic examples and no costly human labels, LongVPO outperforms state-of-the-art open-source models on multiple long-video benchmarks.
  arXiv  Detail & Related papers  (2026-02-02T17:03:37Z)
• VC-Bench: Pioneering the Video Connecting Benchmark with a Dataset and Evaluation Metrics [83.61875204972465]
  We introduce Video Connecting, a task that aims to generate smooth intermediate video content between given start and end clips.<n>To bridge this gap, we proposed VC-Bench, a novel benchmark specifically designed for video connecting.<n> VC-Bench focuses on three core aspects: Video Quality Score VQS, Start-End Consistency Score SECS, and Transition Smoothness Score TSS.
  arXiv  Detail & Related papers  (2026-01-27T06:15:12Z)
• VIPER: Process-aware Evaluation for Generative Video Reasoning [64.86465792516658]
  We introduce VIPER, a comprehensive benchmark spanning 16 tasks across temporal, structural, symbolic, spatial, physics, and planning reasoning.<n>Our experiments reveal that state-of-the-art video models achieve only about 20% POC@1.0 and exhibit a significant outcome-hacking.
  arXiv  Detail & Related papers  (2025-12-31T16:31:59Z)
• SVBench: A Benchmark with Temporal Multi-Turn Dialogues for Streaming Video Understanding [56.78088668917983]
  We introduce SVBench, a pioneering benchmark with temporal multi-turn question-answering chains.<n>We design a semi-automated annotation pipeline to obtain 49,979 Question-Answer (QA) pairs of 1,353 streaming videos.<n>Our experimental results, obtained from 14 models in dialogue and streaming evaluations, reveal that while the closed-source GPT-4o outperforms others, most open-source LVLMs struggle with long-context streaming video understanding.
  arXiv  Detail & Related papers  (2025-02-15T14:29:44Z)
• CaReBench: A Fine-Grained Benchmark for Video Captioning and Retrieval [24.203328970223527]
  We present CaReBench, a testing benchmark for fine-grained video captioning and retrieval.<n>Uniquely, it provides manually separated spatial annotations and temporal annotations for each video.<n>Based on this design, we introduce two evaluation metrics, ReBias and CapST, specifically tailored for video retrieval and video captioning tasks.
  arXiv  Detail & Related papers  (2024-12-31T15:53:50Z)
• EvalCrafter: Benchmarking and Evaluating Large Video Generation Models [70.19437817951673]
  We argue that it is hard to judge the large conditional generative models from the simple metrics since these models are often trained on very large datasets with multi-aspect abilities. Our approach involves generating a diverse and comprehensive list of 700 prompts for text-to-video generation. Then, we evaluate the state-of-the-art video generative models on our carefully designed benchmark, in terms of visual qualities, content qualities, motion qualities, and text-video alignment with 17 well-selected objective metrics.
  arXiv  Detail & Related papers  (2023-10-17T17:50:46Z)
• OnlineRefer: A Simple Online Baseline for Referring Video Object Segmentation [75.07460026246582]
  Referring video object segmentation (RVOS) aims at segmenting an object in a video following human instruction. Current state-of-the-art methods fall into an offline pattern, in which each clip independently interacts with text embedding. We propose a simple yet effective online model using explicit query propagation, named OnlineRefer.
  arXiv  Detail & Related papers  (2023-07-18T15:43:35Z)
• A CLIP-Hitchhiker's Guide to Long Video Retrieval [84.36155238161462]
  We study the adaptation of image-text models for long video retrieval. Recent works have demonstrated state-of-the-art performance in video retrieval by adopting CLIP. We find that the simple yet effective baseline of weighted-mean of frame embeddings via query-scoring is a significant improvement.
  arXiv  Detail & Related papers  (2022-05-17T17:26:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.