RefVideo-6M: A Reliable Reference-Based Dataset for Instructional Video Editing

• URL: http://arxiv.org/abs/2608.26101v1
• Date: Wed, 26 Aug 2026 17:59:19 GMT
• Title: RefVideo-6M: A Reliable Reference-Based Dataset for Instructional Video Editing
• Abstract summary: RefVideo-6M is a large-scale reference-guided editing dataset containing 5 million video editing samples and 1 million image editing samples.<n>To ensure reliable supervision, our dataset uses a construction pipeline that treats artifact-free real videos as editing targets.<n>It provides approximately 6 million visual references, covering diverse reference types and editing scenarios.
• Score: 66.83679558829465
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Recent advances in video editing have been largely driven by large-scale instruction-based datasets. However, existing datasets still suffer from two critical limitations. First, target videos are commonly produced by automatic editing models, which may introduce visible artifacts and unreliable supervision signals. Second, most public datasets rely primarily on textual instructions, while lacking visual references that are crucial for precise, identity-preserving, and controllable editing. To address these limitations, we introduce RefVideo-6M, a large-scale reference-guided editing dataset containing 5 million video editing samples and 1 million image editing samples. To ensure reliable supervision, our dataset uses a construction pipeline that treats artifact-free real videos as editing targets and generates quality-filtered input conditions with multiple editing experts. In addition, it provides approximately 6 million visual references, covering diverse reference types and editing scenarios, thereby enabling models to learn fine-grained visual correspondence beyond text-only instructions. Based on RefVideo-6M, we further train a reference-guided video editing model, Ref-MoT, to evaluate the effectiveness and scalability of the proposed dataset. Extensive experiments demonstrate that RefVideo-6M provides substantially more reliable supervision than existing datasets and enables the training of powerful editing models with improved visual quality, controllability, and reference consistency. The open-source dataset is available at https://huggingface.co/datasets/RefVideo6M/RefVideo6M.

Related papers

• OmniEdit-Bench: A Comprehensive Benchmark for Instruction-based Video Editing [53.21868575804758]
  We introduce a comprehensive and structured benchmark for instruction-based video editing (IVE)<n>Our benchmark decomposes editing tasks into multiple video-specific dimensions, including spatial, temporal, audio, and reference-based editing.<n>We propose an evaluation framework that assesses editing quality from four complementary dimensions: accuracy, preservation, realism, and consistency.
  arXiv  Detail & Related papers  (2026-08-05T16:58:48Z)
• CoVEBench: Can Video Editing Models Handle Complex Instructions? [26.871333176496126]
  CoVEBench is a compositional video editing benchmark comprising 416 curated source videos, 626 multi-point editing instructions, and 9,990 fine-grained checklist items.<n>It evaluates models via MLLM-judged instruction compliance and video fidelity, alongside automated metrics for video quality.
  arXiv  Detail & Related papers  (2026-06-07T02:29:41Z)
• JAVEDIT: Joint Audio-Visual Instruction-Guided Video Editing with Agentic Data Curation [100.46371655184585]
  JAVEdit-100k is the first large-scale, high-quality dataset tailored for instruction-guided joint audio-visual editing.<n>This dataset is rigorously constructed via four meticulously designed generation pipelines, seamlessly paired with an agent-in-the-loop quality control mechanism.<n>To address the lack of standardized evaluation within the field, we introduce JAVEditBench, a comprehensive benchmark featuring curated source videos and human-aligned instructions.
  arXiv  Detail & Related papers  (2026-06-02T05:26:06Z)
• LIVE: Leveraging Image Manipulation Priors for Instruction-based Video Editing [45.1196749483108]
  We propose LIVE, a joint training framework that leverages large-scale, high-quality image editing data alongside video datasets to bolster editing capabilities.<n>We introduce a frame-wise token noise strategy, which treats the latents of specific frames as reasoning tokens, leveraging large pretrained video generative models to create plausible temporal transformations.<n>We curate a comprehensive evaluation benchmark encompassing over 60 challenging tasks that are prevalent in image editing but scarce in existing video datasets.
  arXiv  Detail & Related papers  (2026-04-18T15:09:01Z)
• Kiwi-Edit: Versatile Video Editing via Instruction and Reference Guidance [55.32799307123252]
  We introduce a scalable data generation pipeline that transforms existing video editing pairs into high-fidelity training quadruplets.<n>We propose a unified editing architecture, Kiwi-Edit, that synergizes learnable queries and latent visual features for reference semantic guidance.
  arXiv  Detail & Related papers  (2026-03-02T18:46:28Z)
• VEGGIE: Instructional Editing and Reasoning Video Concepts with Grounded Generation [70.87745520234012]
  We introduce VEGGIE, a simple end-to-end framework that unifies video concept editing, grounding, and reasoning based on diverse user instructions.<n> VEGGIE shows strong performance in instructional video editing with different editing skills, outperforming the best instructional baseline as a versatile model.
  arXiv  Detail & Related papers  (2025-03-18T15:31:12Z)
• VIVID-10M: A Dataset and Baseline for Versatile and Interactive Video Local Editing [13.006616304789878]
  This paper introduces a dataset VIVID-10M and a baseline model VIVID.<n>VIVID-10M is the first large-scale hybrid image-video local editing dataset.<n>Our approach achieves state-of-the-art performance in video local editing, surpassing baseline methods in both automated metrics and user studies.
  arXiv  Detail & Related papers  (2024-11-22T10:04:05Z)
• VLKEB: A Large Vision-Language Model Knowledge Editing Benchmark [53.091690659399234]
  knowledge editing on large language models (LLMs) has received considerable attention. The existing LVLM editing benchmark, which comprises three metrics (Reliability, Locality, and Generality), falls short in the quality of synthesized evaluation images. We employ more reliable data collection methods to construct a new Large $textbfV$ision-$textbfL$anguage Model.
  arXiv  Detail & Related papers  (2024-03-12T06:16:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.