JAVEDIT: Joint Audio-Visual Instruction-Guided Video Editing with Agentic Data Curation

• URL: http://arxiv.org/abs/2606.03168v1
• Date: Tue, 02 Jun 2026 05:26:06 GMT
• Title: JAVEDIT: Joint Audio-Visual Instruction-Guided Video Editing with Agentic Data Curation
• Abstract summary: JAVEdit-100k is the first large-scale, high-quality dataset tailored for instruction-guided joint audio-visual editing.<n>This dataset is rigorously constructed via four meticulously designed generation pipelines, seamlessly paired with an agent-in-the-loop quality control mechanism.<n>To address the lack of standardized evaluation within the field, we introduce JAVEditBench, a comprehensive benchmark featuring curated source videos and human-aligned instructions.
• Score: 100.46371655184585
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While instruction-based video editing has seen significant progress, joint audio-visual editing remains constrained by the absence of dedicated datasets and benchmarks. To bridge this gap, we present JAVEdit-100k, the first large-scale, high-quality dataset tailored for instruction-guided joint audio-visual editing. Focusing on human-centric videos, JAVEdit-100k comprises approximately 100K editing triplets spanning five distinct categories, including subject editing and speech editing. This dataset is rigorously constructed via four meticulously designed generation pipelines, seamlessly paired with an agent-in-the-loop quality control mechanism. Furthermore, to address the lack of standardized evaluation within the field, we introduce JAVEditBench, a comprehensive benchmark featuring curated source videos and human-aligned instructions across all editing categories. Finally, we propose JAVEdit, a pioneering baseline model for instruction-guided joint audio-visual editing. Experiments show that \model\ outperforms all baselines on five of six evaluation metrics.

Related papers

• RefVideo-6M: A Reliable Reference-Based Dataset for Instructional Video Editing [66.83679558829465]
  RefVideo-6M is a large-scale reference-guided editing dataset containing 5 million video editing samples and 1 million image editing samples.<n>To ensure reliable supervision, our dataset uses a construction pipeline that treats artifact-free real videos as editing targets.<n>It provides approximately 6 million visual references, covering diverse reference types and editing scenarios.
  arXiv  Detail & Related papers  (2026-08-26T17:59:19Z)
• OmniEdit-Bench: A Comprehensive Benchmark for Instruction-based Video Editing [53.21868575804758]
  We introduce a comprehensive and structured benchmark for instruction-based video editing (IVE)<n>Our benchmark decomposes editing tasks into multiple video-specific dimensions, including spatial, temporal, audio, and reference-based editing.<n>We propose an evaluation framework that assesses editing quality from four complementary dimensions: accuracy, preservation, realism, and consistency.
  arXiv  Detail & Related papers  (2026-08-05T16:58:48Z)
• AVE-Compass: Towards Holistic Evaluation for Audio-Video Editing Abilities [38.02927226790559]
  We introduce AVE-, a comprehensive benchmark with 145 curated videos, 196 audio-visually coupled editing instructions, and 2,688 fine-grained checklist items.<n>It evaluates Instruction Following, Fidelity Preserving, Realism, and Editing Intent through checklist-based MLLM judging and a dedicated realism, complemented by automated cross-modal, video, and audio metrics.<n>We propose AVE-Agent, a modular evaluator framework that decomposes complex instructions into dependent subtasks and iteratively improves editing results through self-reflection and feedback.
  arXiv  Detail & Related papers  (2026-07-17T12:40:13Z)
• CoVEBench: Can Video Editing Models Handle Complex Instructions? [26.871333176496126]
  CoVEBench is a compositional video editing benchmark comprising 416 curated source videos, 626 multi-point editing instructions, and 9,990 fine-grained checklist items.<n>It evaluates models via MLLM-judged instruction compliance and video fidelity, alongside automated metrics for video quality.
  arXiv  Detail & Related papers  (2026-06-07T02:29:41Z)
• VEFX-Bench: A Holistic Benchmark for Generic Video Editing and Visual Effects [34.23919867305323]
  We introduce VEFX-Dataset, a human-annotated dataset containing 5,049 video editing examples.<n> VEFX-Reward is a reward model designed specifically for video editing quality assessment.<n> VEFX-Bench is a benchmark of 300 curated video-prompt pairs for standardized comparison of editing systems.
  arXiv  Detail & Related papers  (2026-04-17T17:28:24Z)
• OpenVE-3M: A Large-Scale High-Quality Dataset for Instruction-Guided Video Editing [40.87442780303236]
  OpenVE-3M is an open-source, large-scale, and high-quality dataset for instruction-based video editing.<n>To address the lack of a unified benchmark in the field, we construct OpenVE-Bench, containing 431 video-edit pairs.<n>We present OpenVE-Edit, a 5B model trained on our dataset that demonstrates remarkable efficiency and effectiveness.
  arXiv  Detail & Related papers  (2025-12-08T18:55:07Z)
• In-Context Learning with Unpaired Clips for Instruction-based Video Editing [51.943707933717185]
  We introduce a low-cost pretraining strategy for instruction-based video editing.<n>Our framework first pretrains on approximately 1M real video clips to learn basic editing concepts.<n>Our method surpasses existing instruction-based video editing approaches in both instruction alignment and visual fidelity.
  arXiv  Detail & Related papers  (2025-10-16T13:02:11Z)
• IVEBench: Modern Benchmark Suite for Instruction-Guided Video Editing Assessment [108.8652018167452]
  IVEBench is a benchmark suite specifically designed for instruction-guided video editing assessment.<n>It comprises a diverse database of 600 high-quality source videos, spanning seven semantic dimensions, and covering video lengths ranging from 32 to 1,024 frames.<n>IVEBench establishes a three-dimensional evaluation protocol encompassing video quality, instruction compliance and video fidelity.
  arXiv  Detail & Related papers  (2025-10-13T17:27:08Z)
• AnyEdit: Mastering Unified High-Quality Image Editing for Any Idea [88.79769371584491]
  We present AnyEdit, a comprehensive multi-modal instruction editing dataset.<n>We ensure the diversity and quality of the AnyEdit collection through three aspects: initial data diversity, adaptive editing process, and automated selection of editing results.<n>Experiments on three benchmark datasets show that AnyEdit consistently boosts the performance of diffusion-based editing models.
  arXiv  Detail & Related papers  (2024-11-24T07:02:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.