VEFX-Bench: A Holistic Benchmark for Generic Video Editing and Visual Effects

• URL: http://arxiv.org/abs/2604.16272v1
• Date: Fri, 17 Apr 2026 17:28:24 GMT
• Title: VEFX-Bench: A Holistic Benchmark for Generic Video Editing and Visual Effects
• Abstract summary: We introduce VEFX-Dataset, a human-annotated dataset containing 5,049 video editing examples across 9 major editing categories and 32 subcategories.<n> VEFX-Reward is a reward model designed specifically for video editing quality assessment.<n> VEFX-Bench is a benchmark of 300 curated video-prompt pairs for standardized comparison of editing systems.
• Score: 33.5974992507736
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As AI-assisted video creation becomes increasingly practical, instruction-guided video editing has become essential for refining generated or captured footage to meet professional requirements. Yet the field still lacks both a large-scale human-annotated dataset with complete editing examples and a standardized evaluator for comparing editing systems. Existing resources are limited by small scale, missing edited outputs, or the absence of human quality labels, while current evaluation often relies on expensive manual inspection or generic vision-language model judges that are not specialized for editing quality. We introduce VEFX-Dataset, a human-annotated dataset containing 5,049 video editing examples across 9 major editing categories and 32 subcategories, each labeled along three decoupled dimensions: Instruction Following, Rendering Quality, and Edit Exclusivity. Building on VEFX-Dataset, we propose VEFX-Reward, a reward model designed specifically for video editing quality assessment. VEFX-Reward jointly processes the source video, the editing instruction, and the edited video, and predicts per-dimension quality scores via ordinal regression. We further release VEFX-Bench, a benchmark of 300 curated video-prompt pairs for standardized comparison of editing systems. Experiments show that VEFX-Reward aligns more strongly with human judgments than generic VLM judges and prior reward models on both standard IQA/VQA metrics and group-wise preference evaluation. Using VEFX-Reward as an evaluator, we benchmark representative commercial and open-source video editing systems, revealing a persistent gap between visual plausibility, instruction following, and edit locality in current models.

Related papers

• OmniEdit-Bench: A Comprehensive Benchmark for Instruction-based Video Editing [53.21868575804758]
  We introduce a comprehensive and structured benchmark for instruction-based video editing (IVE)<n>Our benchmark decomposes editing tasks into multiple video-specific dimensions, including spatial, temporal, audio, and reference-based editing.<n>We propose an evaluation framework that assesses editing quality from four complementary dimensions: accuracy, preservation, realism, and consistency.
  arXiv  Detail & Related papers  (2026-08-05T16:58:48Z)
• CoVEBench: Can Video Editing Models Handle Complex Instructions? [26.871333176496126]
  CoVEBench is a compositional video editing benchmark comprising 416 curated source videos, 626 multi-point editing instructions, and 9,990 fine-grained checklist items.<n>It evaluates models via MLLM-judged instruction compliance and video fidelity, alongside automated metrics for video quality.
  arXiv  Detail & Related papers  (2026-06-07T02:29:41Z)
• JAVEDIT: Joint Audio-Visual Instruction-Guided Video Editing with Agentic Data Curation [100.46371655184585]
  JAVEdit-100k is the first large-scale, high-quality dataset tailored for instruction-guided joint audio-visual editing.<n>This dataset is rigorously constructed via four meticulously designed generation pipelines, seamlessly paired with an agent-in-the-loop quality control mechanism.<n>To address the lack of standardized evaluation within the field, we introduce JAVEditBench, a comprehensive benchmark featuring curated source videos and human-aligned instructions.
  arXiv  Detail & Related papers  (2026-06-02T05:26:06Z)
• LIVE: Leveraging Image Manipulation Priors for Instruction-based Video Editing [45.1196749483108]
  We propose LIVE, a joint training framework that leverages large-scale, high-quality image editing data alongside video datasets to bolster editing capabilities.<n>We introduce a frame-wise token noise strategy, which treats the latents of specific frames as reasoning tokens, leveraging large pretrained video generative models to create plausible temporal transformations.<n>We curate a comprehensive evaluation benchmark encompassing over 60 challenging tasks that are prevalent in image editing but scarce in existing video datasets.
  arXiv  Detail & Related papers  (2026-04-18T15:09:01Z)
• UniEditBench: A Unified and Cost-Effective Benchmark for Image and Video Editing via Distilled MLLMs [19.634771714749544]
  We present UniEditBench, a unified benchmark for image and video editing.<n>UniEditBench supports reconstruction-based and instruction-driven methods under a shared protocol.<n>We distill a high-capacity MLLM judge into lightweight 4B/8B evaluators that provide multi-dimensional scoring over structural fidelity, text alignment, background consistency, naturalness, and temporal-spatial consistency for videos.
  arXiv  Detail & Related papers  (2026-04-17T09:21:48Z)
• Evaluating Image Editing with LLMs: A Comprehensive Benchmark and Intermediate-Layer Probing Approach [48.01137214342501]
  We introduce TIEdit, a benchmark for systematic evaluation of text-guided image editing methods.<n>To obtain reliable subjective ratings, 20 experts are recruited to produce 307,200 raw subjective ratings.<n>We further propose EditProbe, an LLM-based evaluator that estimates editing quality via intermediate-layer probing of hidden representations.
  arXiv  Detail & Related papers  (2026-03-20T09:05:10Z)
• IVEBench: Modern Benchmark Suite for Instruction-Guided Video Editing Assessment [108.8652018167452]
  IVEBench is a benchmark suite specifically designed for instruction-guided video editing assessment.<n>It comprises a diverse database of 600 high-quality source videos, spanning seven semantic dimensions, and covering video lengths ranging from 32 to 1,024 frames.<n>IVEBench establishes a three-dimensional evaluation protocol encompassing video quality, instruction compliance and video fidelity.
  arXiv  Detail & Related papers  (2025-10-13T17:27:08Z)
• EditVerse: Unifying Image and Video Editing and Generation with In-Context Learning [58.53074381801114]
  We introduce EditVerse, a unified framework for image and video generation and editing within a single model.<n>By representing all modalities, i.e. text, image, and video, as a unified token sequence, EditVerse leverages self-attention to achieve robust in-context learning.<n>We present EditVerseBench, the first benchmark for instruction-based video editing covering diverse tasks and resolutions.
  arXiv  Detail & Related papers  (2025-09-24T17:59:30Z)
• EditDuet: A Multi-Agent System for Video Non-Linear Editing [24.334561615501105]
  We propose to automate the core task of video editing, formulating it as sequential decision making process.<n>The Editor takes as input a collection of video clips together with natural language instructions and uses tools commonly found in video editing software to produce an edited sequence.<n>We evaluate our system's output video sequences qualitatively and quantitatively through a user study and find that our system vastly outperforms existing approaches in terms of coverage, time constraint satisfaction, and human preference.
  arXiv  Detail & Related papers  (2025-09-13T00:27:02Z)
• EditBoard: Towards a Comprehensive Evaluation Benchmark for Text-Based Video Editing Models [16.045012576543474]
  Text-based video editing has emerged as a promising field, enabling precise modifications to videos based on text prompts.<n>Existing evaluations are limited and inconsistent, typically summarizing overall performance with a single score.<n>We propose EditBoard, the first comprehensive evaluation benchmark for text-based video editing models.
  arXiv  Detail & Related papers  (2024-09-15T08:43:18Z)
• VE-Bench: Subjective-Aligned Benchmark Suite for Text-Driven Video Editing Quality Assessment [6.627422081288281]
  We introduce VE-Bench, a benchmark suite tailored to the assessment of text-driven video editing.<n>This suite includes VE-Bench DB, a video quality assessment (VQA) database for video editing.<n> VE-Bench QA focuses on the text-video alignment and the relevance modeling between source and edited videos.
  arXiv  Detail & Related papers  (2024-08-21T09:49:32Z)
• Beyond Raw Videos: Understanding Edited Videos with Large Multimodal Model [62.38322742493649]
  We build a video VQA benchmark covering editing categories, i.e., effect, funny, meme, and game. Most of the open-source video LMMs perform poorly on the benchmark, indicating a huge domain gap between edited short videos on social media and regular raw videos. To improve the generalization ability of LMMs, we collect a training set for the proposed benchmark based on both Panda-70M/WebVid raw videos and small-scale TikTok/CapCut edited videos.
  arXiv  Detail & Related papers  (2024-06-15T03:28:52Z)
• VLKEB: A Large Vision-Language Model Knowledge Editing Benchmark [53.091690659399234]
  knowledge editing on large language models (LLMs) has received considerable attention. The existing LVLM editing benchmark, which comprises three metrics (Reliability, Locality, and Generality), falls short in the quality of synthesized evaluation images. We employ more reliable data collection methods to construct a new Large $textbfV$ision-$textbfL$anguage Model.
  arXiv  Detail & Related papers  (2024-03-12T06:16:33Z)
• The Anatomy of Video Editing: A Dataset and Benchmark Suite for AI-Assisted Video Editing [90.59584961661345]
  This work introduces the Anatomy of Video Editing, a dataset, and benchmark, to foster research in AI-assisted video editing. Our benchmark suite focuses on video editing tasks, beyond visual effects, such as automatic footage organization and assisted video assembling. To enable research on these fronts, we annotate more than 1.5M tags, with relevant concepts to cinematography, from 196176 shots sampled from movie scenes.
  arXiv  Detail & Related papers  (2022-07-20T10:53:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.