OmniEdit-Bench: A Comprehensive Benchmark for Instruction-based Video Editing
- URL: http://arxiv.org/abs/2608.05049v1
- Date: Wed, 05 Aug 2026 16:58:48 GMT
- Title: OmniEdit-Bench: A Comprehensive Benchmark for Instruction-based Video Editing
- Abstract summary: We introduce a comprehensive and structured benchmark for instruction-based video editing (IVE)<n>Our benchmark decomposes editing tasks into multiple video-specific dimensions, including spatial, temporal, audio, and reference-based editing.<n>We propose an evaluation framework that assesses editing quality from four complementary dimensions: accuracy, preservation, realism, and consistency.
- Score: 53.21868575804758
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Instruction-based video editing (IVE) is an emerging field with broad applications, yet evaluating editing models remains challenging. Existing benchmarks suffer from two major limitations: limited task coverage inherited from image editing, which overlooks video-specific dimensions, and inadequate metrics that fail to measure instruction fidelity, allowing incorrect edits to receive high scores due to strong visual priors from the original video. To address these issues, we introduce a comprehensive and structured benchmark for IVE. Our benchmark decomposes editing tasks into multiple video-specific dimensions, including spatial, temporal, audio, and reference-based editing, extending beyond conventional frame-level evaluation. It also distinguishes explicit and implicit instructions and incorporates reasoning-based scenarios to better reflect real-world requirements. Furthermore, we propose an evaluation framework that assesses editing quality from four complementary dimensions: accuracy, preservation, realism, and consistency, using both human judgments and state-of-the-art vision-language models. To emphasize instruction fidelity, we introduce an accuracy-aware penalty mechanism that conditions other scores on accuracy, preventing visually plausible but incorrect edits from receiving inflated evaluations. Extensive experiments on representative open-source and commercial models show that current IVE models remain far from satisfactory. OmniEdit-Bench provides a comprehensive and reliable testbed for evaluating instruction-based video editing and offers insights into future research directions.
Related papers
- RefVideo-6M: A Reliable Reference-Based Dataset for Instructional Video Editing [66.83679558829465]
RefVideo-6M is a large-scale reference-guided editing dataset containing 5 million video editing samples and 1 million image editing samples.<n>To ensure reliable supervision, our dataset uses a construction pipeline that treats artifact-free real videos as editing targets.<n>It provides approximately 6 million visual references, covering diverse reference types and editing scenarios.
arXiv Detail & Related papers (2026-08-26T17:59:19Z) - CoVEBench: Can Video Editing Models Handle Complex Instructions? [26.871333176496126]
CoVEBench is a compositional video editing benchmark comprising 416 curated source videos, 626 multi-point editing instructions, and 9,990 fine-grained checklist items.<n>It evaluates models via MLLM-judged instruction compliance and video fidelity, alongside automated metrics for video quality.
arXiv Detail & Related papers (2026-06-07T02:29:41Z) - VEFX-Bench: A Holistic Benchmark for Generic Video Editing and Visual Effects [34.23919867305323]
We introduce VEFX-Dataset, a human-annotated dataset containing 5,049 video editing examples.<n> VEFX-Reward is a reward model designed specifically for video editing quality assessment.<n> VEFX-Bench is a benchmark of 300 curated video-prompt pairs for standardized comparison of editing systems.
arXiv Detail & Related papers (2026-04-17T17:28:24Z) - GEditBench v2: A Human-Aligned Benchmark for General Image Editing [58.86807672117726]
We introduce GEditBench v2, a comprehensive benchmark with 1,200 real-world user queries spanning 23 tasks.<n>We also propose PVC-Judge, an open-source pairwise assessment model for visual consistency, trained via two novel region-decoupled preference data synthesis pipelines.<n> PVC-Judge achieves state-of-the-art evaluation performance among open-source models and even surpasses GPT-5.1 on average.
arXiv Detail & Related papers (2026-03-30T15:08:32Z) - CoEditor++: Instruction-based Visual Editing via Cognitive Reasoning [98.98349220451216]
CoEditor++ is a training-free framework that decomposes editing into "what to edit" and "how to edit"<n>We show that CoEditor++ achieves state-of-the-art performance in both general editing and responsible editing tasks.<n>Our findings suggest the potential toward cognitive-centric instruction-based image editing.
arXiv Detail & Related papers (2026-01-31T12:20:46Z) - ReViSE: Towards Reason-Informed Video Editing in Unified Models with Self-Reflective Learning [57.08352504712699]
Video unified models exhibit strong capabilities in understanding and generation, yet they struggle with reason-informed visual editing.<n>We introduce the Reason-Informed Video Editing (RVE) task, which requires reasoning about physical plausibility and causal dynamics during editing.<n>We propose ReViSE, a framework that unifies generation and evaluation within a single architecture.
arXiv Detail & Related papers (2025-12-10T18:57:09Z) - IVEBench: Modern Benchmark Suite for Instruction-Guided Video Editing Assessment [108.8652018167452]
IVEBench is a benchmark suite specifically designed for instruction-guided video editing assessment.<n>It comprises a diverse database of 600 high-quality source videos, spanning seven semantic dimensions, and covering video lengths ranging from 32 to 1,024 frames.<n>IVEBench establishes a three-dimensional evaluation protocol encompassing video quality, instruction compliance and video fidelity.
arXiv Detail & Related papers (2025-10-13T17:27:08Z) - EditBoard: Towards a Comprehensive Evaluation Benchmark for Text-Based Video Editing Models [16.045012576543474]
Text-based video editing has emerged as a promising field, enabling precise modifications to videos based on text prompts.<n>Existing evaluations are limited and inconsistent, typically summarizing overall performance with a single score.<n>We propose EditBoard, the first comprehensive evaluation benchmark for text-based video editing models.
arXiv Detail & Related papers (2024-09-15T08:43:18Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.