AVE-Compass: Towards Holistic Evaluation for Audio-Video Editing Abilities

• URL: http://arxiv.org/abs/2607.24821v1
• Date: Fri, 17 Jul 2026 12:40:13 GMT
• Title: AVE-Compass: Towards Holistic Evaluation for Audio-Video Editing Abilities
• Abstract summary: We introduce AVE-, a comprehensive benchmark with 145 curated videos, 196 audio-visually coupled editing instructions, and 2,688 fine-grained checklist items.<n>It evaluates Instruction Following, Fidelity Preserving, Realism, and Editing Intent through checklist-based MLLM judging and a dedicated realism, complemented by automated cross-modal, video, and audio metrics.<n>We propose AVE-Agent, a modular evaluator framework that decomposes complex instructions into dependent subtasks and iteratively improves editing results through self-reflection and feedback.
• Score: 38.02927226790559
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While instruction-based video editing has advanced rapidly, real-world videos contain tightly coupled audio and visual signals, and editing one modality often requires coordinated changes in the other. Existing benchmarks primarily evaluate visual transformations on silent clips or isolated audio editing, leaving complex audio-visual editing and cross-modal consistency underexplored. We introduce AVE-Compass, a comprehensive benchmark with 145 curated source videos, 196 audio-visually coupled editing instructions, and 2,688 fine-grained checklist items. It evaluates Instruction Following, Fidelity Preserving, Realism, and Editing Intent through checklist-based MLLM judging and a dedicated realism rubric, complemented by automated cross-modal, video, and audio metrics. Extensive evaluation shows that state-of-the-art models still struggle to execute cross-modal instructions while preserving non-target content. We further propose AVE-Agent, a modular agent framework that decomposes complex instructions into dependent subtasks and iteratively improves editing results through self-reflection and evaluator feedback. AVE-Agent improves instruction execution, Fidelity Preserving, and audio-visual alignment in joint editing while maintaining competitive perceptual quality.

Related papers

• AVENUE: Audio-Video EditiNg Understanding and Evaluation [16.21802697462669]
  We introduce AVENUE, Audio-Video EditiNg Understanding and Evaluation.<n>It is a benchmark and modality-aware evaluation framework to drive progress toward controllable AV editing models.<n>Our findings reveal a fundamental open challenge: when editing one modality, existing models frequently induce unintended changes in the other, regardless of paradigm.
  arXiv  Detail & Related papers  (2026-08-31T13:05:01Z)
• OmniEdit-Bench: A Comprehensive Benchmark for Instruction-based Video Editing [53.21868575804758]
  We introduce a comprehensive and structured benchmark for instruction-based video editing (IVE)<n>Our benchmark decomposes editing tasks into multiple video-specific dimensions, including spatial, temporal, audio, and reference-based editing.<n>We propose an evaluation framework that assesses editing quality from four complementary dimensions: accuracy, preservation, realism, and consistency.
  arXiv  Detail & Related papers  (2026-08-05T16:58:48Z)
• Hybrid Diffusion Transformer for Instruction-Guided Audio Editing via Rectified Flow [21.976898083812674]
  We propose a hybrid two-stage diffusion transformer architecture for instruction-guided audio editing.<n>It performs joint attention over audio and text tokens to establish coarse semantic alignment at low-resolution stage, then switches to alternating joint-attention and cross-attention blocks to refine editing details at high-resolution stage.
  arXiv  Detail & Related papers  (2026-06-18T11:20:08Z)
• CoVEBench: Can Video Editing Models Handle Complex Instructions? [26.871333176496126]
  CoVEBench is a compositional video editing benchmark comprising 416 curated source videos, 626 multi-point editing instructions, and 9,990 fine-grained checklist items.<n>It evaluates models via MLLM-judged instruction compliance and video fidelity, alongside automated metrics for video quality.
  arXiv  Detail & Related papers  (2026-06-07T02:29:41Z)
• JAVEDIT: Joint Audio-Visual Instruction-Guided Video Editing with Agentic Data Curation [100.46371655184585]
  JAVEdit-100k is the first large-scale, high-quality dataset tailored for instruction-guided joint audio-visual editing.<n>This dataset is rigorously constructed via four meticulously designed generation pipelines, seamlessly paired with an agent-in-the-loop quality control mechanism.<n>To address the lack of standardized evaluation within the field, we introduce JAVEditBench, a comprehensive benchmark featuring curated source videos and human-aligned instructions.
  arXiv  Detail & Related papers  (2026-06-02T05:26:06Z)
• Towards Universal Video MLLMs with Attribute-Structured and Quality-Verified Instructions [74.27249614046309]
  ASID-1M is an open-source collection of one million structured, fine-grained audiovisual instruction annotations.<n>ASID-Verify is a scalable data curation pipeline for annotation.<n>ASID-Captioner is a video understanding model trained via Supervised Fine-Tuning.
  arXiv  Detail & Related papers  (2026-02-13T15:20:54Z)
• Schrodinger Audio-Visual Editor: Object-Level Audiovisual Removal [90.14887235360611]
  SAVEBench is a paired audiovisual dataset with text and mask conditions to enable object-grounded source-to-target learning.<n>SAVE incorporates a Schrodinger Bridge that learns a direct transport from source to target audiovisual mixtures.<n>Our evaluation demonstrates that the proposed SAVE model is able to remove the target objects in audio and visual content while preserving the remaining content.
  arXiv  Detail & Related papers  (2025-12-14T23:19:15Z)
• Audio-sync Video Instance Editing with Granularity-Aware Mask Refiner [66.96392168346851]
  AVI-Edit is a framework for audio-sync video instance editing.<n>We propose a granularity-aware mask refiner that iteratively refines coarse user-provided masks into precise instance-level regions.<n>We also design a self-feedback audio agent to curate high-quality audio guidance, providing fine-grained temporal control.
  arXiv  Detail & Related papers  (2025-12-11T11:58:53Z)
• AV-Edit: Multimodal Generative Sound Effect Editing via Audio-Visual Semantic Joint Control [10.55114688654566]
  AV-Edit is a generative sound effect editing framework that enables fine-grained editing of existing audio tracks in videos.<n>The proposed method employs a specially designed contrastive audio-visual masking autoencoder (CAV-MAE-Edit) for multimodal pre-training.<n> Experiments demonstrate that the proposed AV-Edit generates high-quality audio with precise modifications based on visual content.
  arXiv  Detail & Related papers  (2025-11-26T07:59:53Z)
• RFM-Editing: Rectified Flow Matching for Text-guided Audio Editing [21.479883699581308]
  We propose a novel end-to-end efficient rectified flow matching-based diffusion framework for audio editing.<n> Experiments show that our model achieves faithful semantic alignment without requiring auxiliary captions or masks.
  arXiv  Detail & Related papers  (2025-09-17T14:13:40Z)
• EditGen: Harnessing Cross-Attention Control for Instruction-Based Auto-Regressive Audio Editing [54.10773655199149]
  We investigate leveraging cross-attention control for efficient audio editing within auto-regressive models.<n>Inspired by image editing methodologies, we develop a Prompt-to-Prompt-like approach that guides edits through cross and self-attention mechanisms.
  arXiv  Detail & Related papers  (2025-07-15T08:44:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.