AVENUE: Audio-Video EditiNg Understanding and Evaluation

• URL: http://arxiv.org/abs/2609.04253v1
• Date: Mon, 31 Aug 2026 13:05:01 GMT
• Title: AVENUE: Audio-Video EditiNg Understanding and Evaluation
• Abstract summary: We introduce AVENUE, Audio-Video EditiNg Understanding and Evaluation.<n>It is a benchmark and modality-aware evaluation framework to drive progress toward controllable AV editing models.<n>Our findings reveal a fundamental open challenge: when editing one modality, existing models frequently induce unintended changes in the other, regardless of paradigm.
• Score: 16.21802697462669
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Audio-video (AV) editing aims to modify audio and video content according to a target prompt. Unlike single-modality editing, AV editing requires models to infer a modality-selective edit scope from the prompt alone: determining not only what should change, but also which modality should be preserved. Faithfully evaluating such models therefore requires both (i) benchmarks that span diverse edit types and modality categories, and (ii) evaluation that is itself modality-aware and sample-specific. However, existing AV editing benchmarks provide limited coverage of edit types and modality combinations, while current evaluation systems are often modality-blind and sample-agnostic, making it difficult to assess whether models faithfully preserve the unintended modality. To address these gaps, we introduce AVENUE, Audio-Video EditiNg Understanding and Evaluation, comprising two contributions: (1) a benchmark of 1,291 source clips and 7,957 editing instructions across audio-targeted, video-targeted, and AV-coupled edit types, curated and human-verified from VGGSound; and (2) a sample-specific, modality-aware evaluation framework that specifies, for each sample, both the intended change and the content that must remain intact. We evaluate representative AV editing models spanning three editing paradigms : joint, sequential, and separate, providing the first systematic analysis of modality-selectivity across paradigms. Our findings reveal a fundamental open challenge: when editing one modality, existing models frequently induce unintended changes in the other, regardless of paradigm. AVENUE provides a benchmark and modality-aware evaluation framework to drive progress toward more controllable AV editing models. Our dataset is publicly available on Hugging Face: https://huggingface.co/datasets/AVENUE-dataset/AVENUE.

Related papers

• OmniEdit-Bench: A Comprehensive Benchmark for Instruction-based Video Editing [53.21868575804758]
  We introduce a comprehensive and structured benchmark for instruction-based video editing (IVE)<n>Our benchmark decomposes editing tasks into multiple video-specific dimensions, including spatial, temporal, audio, and reference-based editing.<n>We propose an evaluation framework that assesses editing quality from four complementary dimensions: accuracy, preservation, realism, and consistency.
  arXiv  Detail & Related papers  (2026-08-05T16:58:48Z)
• AVE-Compass: Towards Holistic Evaluation for Audio-Video Editing Abilities [38.02927226790559]
  We introduce AVE-, a comprehensive benchmark with 145 curated videos, 196 audio-visually coupled editing instructions, and 2,688 fine-grained checklist items.<n>It evaluates Instruction Following, Fidelity Preserving, Realism, and Editing Intent through checklist-based MLLM judging and a dedicated realism, complemented by automated cross-modal, video, and audio metrics.<n>We propose AVE-Agent, a modular evaluator framework that decomposes complex instructions into dependent subtasks and iteratively improves editing results through self-reflection and feedback.
  arXiv  Detail & Related papers  (2026-07-17T12:40:13Z)
• Taming I2V models for Image HOI Editing: A Cognitive Benchmark and Agentic Self-Correcting Framework [62.705866028818576]
  Current image editing methods excel at static attributes but fail at complex Human-Object Interactions (HOI)<n>We introduce HOI-Edit, a comprehensive benchmark with three progressive cognitive levels.<n>On HOI-Edit, SCPE achieves performance competitive with state-of-the-art (SOTA) editing models like Nano Banana on interaction.
  arXiv  Detail & Related papers  (2026-06-17T13:44:26Z)
• MMAE: A Massive Multitask Audio Editing Benchmark [66.74229858407413]
  MMAE is the first comprehensive evaluation testbed designed for general-purpose instruction-based audio editing.<n>It comprises 2,000 high-fidelity samples paired with a pioneering rubric-based evaluation framework.<n>Our evaluation reveals that current systems remain far from achieving reliable edits.
  arXiv  Detail & Related papers  (2026-06-05T12:52:41Z)
• MSAVBench: Towards Comprehensive and Reliable Evaluation of Multi-Shot Audio-Video Generation [88.7702943548674]
  We introduce MSAVBench, the first comprehensive benchmark and adaptive hybrid evaluation framework for multi-shot audio-video generation.<n>Our benchmark spans four key dimensions, video, audio, shot, and reference, covering diverse task settings, varying shot counts of up to 15, and challenging non-realistic scenarios.<n> MSAVBench achieves high alignment with human judgments, reaching a Spearman rank correlation of 91.5%.
  arXiv  Detail & Related papers  (2026-05-19T17:59:33Z)
• AVGen-Bench: A Task-Driven Benchmark for Multi-Granular Evaluation of Text-to-Audio-Video Generation [42.157453071865056]
  AVGen-Bench is a task-driven benchmark for T2AV generation featuring high-quality prompts across 11 real-world categories.<n>We propose a multi-granular evaluation framework that combines lightweight specialist models with Multimodal Large Language Models (MLLMs)<n>Our evaluation reveals a pronounced gap between strong audio-visual aesthetics and weak semantic reliability, including persistent failures in text rendering, speech coherence, physical reasoning, and a universal breakdown in musical pitch control.
  arXiv  Detail & Related papers  (2026-04-09T17:59:39Z)
• ReViSE: Towards Reason-Informed Video Editing in Unified Models with Self-Reflective Learning [57.08352504712699]
  Video unified models exhibit strong capabilities in understanding and generation, yet they struggle with reason-informed visual editing.<n>We introduce the Reason-Informed Video Editing (RVE) task, which requires reasoning about physical plausibility and causal dynamics during editing.<n>We propose ReViSE, a framework that unifies generation and evaluation within a single architecture.
  arXiv  Detail & Related papers  (2025-12-10T18:57:09Z)
• EdiVal-Agent: An Object-Centric Framework for Automated, Fine-Grained Evaluation of Multi-Turn Editing [170.71134330650796]
  EdiVal-Agent is an object-centric evaluation framework for instruction-based image editing.<n>It is designed to assess not only standard single-turn but also multi-turn instruction-based editing with precision.<n>We build EdiVal-Bench, a benchmark covering 9 instruction types and 13 state-of-the-art editing models spanning in-context, flow-matching, and diffusion paradigms.
  arXiv  Detail & Related papers  (2025-09-16T17:45:39Z)
• EditBoard: Towards a Comprehensive Evaluation Benchmark for Text-Based Video Editing Models [16.045012576543474]
  Text-based video editing has emerged as a promising field, enabling precise modifications to videos based on text prompts.<n>Existing evaluations are limited and inconsistent, typically summarizing overall performance with a single score.<n>We propose EditBoard, the first comprehensive evaluation benchmark for text-based video editing models.
  arXiv  Detail & Related papers  (2024-09-15T08:43:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.