VicEdit: Learning to Edit Videos from Visual In-Context Examples

• URL: http://arxiv.org/abs/2608.16745v1
• Date: Mon, 17 Aug 2026 15:55:40 GMT
• Title: VicEdit: Learning to Edit Videos from Visual In-Context Examples
• Abstract summary: We propose Visual In-context Editing, a new paradigm elevating video editing from textual instructions to multi-modal visual guidance.<n>We generate VicEdit-400K, the first large-scale dataset for visual in-context video editing.<n>VicEdit achieves state-of-the-art performance across both basic instruction editing and visual in-context editing tasks.
• Score: 122.14105395087746
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite progress in instruction-based video editing, unimodal textual instructions inherently struggle to convey fine-grained textures and complex dynamics. To bridge this perceptual gap, we propose Visual In-context Editing, a new paradigm elevating video editing from textual instructions to multi-modal visual guidance encompassing single image, image pair, and video pair. To facilitate this paradigm, we curate VicEdit-400K, the first large-scale dataset for visual in-context video editing. We develop an automated pipeline to generate 400K high-quality samples across ten task types, ensuring superior visual fidelity and semantic consistency through multi-dimensional filtering. Leveraging this foundation, we introduce VicEdit, a unified framework to bridge visual and textual contexts. To adaptively extract editing semantics from heterogeneous references, we design Modality-Adaptive Semantic Distillation, which produces modality-specific semantic tokens from visual references. These tokens are then synergistically integrated with textual instructions through Dual-Context Injection, enabling the generation process to benefit from both visual and textual signals. Extensive evaluations on VicEditBench demonstrate that VicEdit achieves state-of-the-art performance across both basic instruction editing and visual in-context editing tasks, establishing visual in-context learning as a powerful and controllable paradigm for video editing.

Related papers

• FlowMimic: Mask-free Visual Editing and Generation with Pixel-pair Warped Flow Field for Online Video Editing Data Generation and Modality Mimicry [3.8989988029821396]
  We explore the integration of both generation and editing capabilities for video and image modalities within a single model.<n>We develop a pixel-pair temporal warped flow field that can directly generate corresponding video editing samples in real time from image editing samples.
  arXiv  Detail & Related papers  (2026-07-20T17:58:03Z)
• Text-Vision Co-Instructed Image Editing [11.968553566506626]
  We present Text-Vision Co-Instructed Image Editing, which models textual instructions as semantic intent and sparse visual instructions as spatial guidance.<n>By integrating semantic intent and spatial constraints, TV-Edit leads to more precise spatial control, less instruction ambiguity, and stronger structural consistency than text-only or drag-based alternatives.
  arXiv  Detail & Related papers  (2026-06-15T14:16:05Z)
• Kiwi-Edit: Versatile Video Editing via Instruction and Reference Guidance [55.32799307123252]
  We introduce a scalable data generation pipeline that transforms existing video editing pairs into high-fidelity training quadruplets.<n>We propose a unified editing architecture, Kiwi-Edit, that synergizes learnable queries and latent visual features for reference semantic guidance.
  arXiv  Detail & Related papers  (2026-03-02T18:46:28Z)
• Text-Driven Reasoning Video Editing via Reinforcement Learning on Digital Twin Representations [8.479321655643195]
  We introduce reasoning video editing, a task where video editing models must interpret implicit queries through multi-hop reasoning to infer editing targets before executing modifications.<n> RIVER decouples reasoning from generation through digital twin representations of video content that preserve spatial relationships, temporal trajectories, and semantic attributes.<n> RIVER training uses reinforcement learning with rewards that evaluate reasoning accuracy and generation quality.
  arXiv  Detail & Related papers  (2025-11-18T03:37:19Z)
• EditVerse: Unifying Image and Video Editing and Generation with In-Context Learning [58.53074381801114]
  We introduce EditVerse, a unified framework for image and video generation and editing within a single model.<n>By representing all modalities, i.e. text, image, and video, as a unified token sequence, EditVerse leverages self-attention to achieve robust in-context learning.<n>We present EditVerseBench, the first benchmark for instruction-based video editing covering diverse tasks and resolutions.
  arXiv  Detail & Related papers  (2025-09-24T17:59:30Z)
• UNIC: Unified In-Context Video Editing [76.76077875564526]
  UNified In-Context Video Editing (UNIC) is a framework that unifies diverse video editing tasks within a single model in an in-context manner.<n>We introduce task-aware RoPE to facilitate consistent temporal positional encoding, and condition bias that enables the model to clearly differentiate different editing tasks.<n>Results demonstrate that our unified approach achieves superior performance on each task and exhibits emergent task composition abilities.
  arXiv  Detail & Related papers  (2025-06-04T17:57:43Z)
• FireEdit: Fine-grained Instruction-based Image Editing via Region-aware Vision Language Model [54.693572837423226]
  FireEdit is an innovative Fine-grained Instruction-based image editing framework that exploits a REgion-aware VLM.<n>FireEdit is designed to accurately comprehend user instructions and ensure effective control over the editing process.<n>Our approach surpasses the state-of-the-art instruction-based image editing methods.
  arXiv  Detail & Related papers  (2025-03-25T16:59:42Z)
• FLATTEN: optical FLow-guided ATTENtion for consistent text-to-video editing [65.60744699017202]
  We introduce optical flow into the attention module in the diffusion model's U-Net to address the inconsistency issue for text-to-video editing. Our method, FLATTEN, enforces the patches on the same flow path across different frames to attend to each other in the attention module. Results on existing text-to-video editing benchmarks show that our proposed method achieves the new state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-10-09T17:59:53Z)
• Visual Instruction Inversion: Image Editing via Visual Prompting [34.96778567507126]
  We present a method for image editing via visual prompting. We leverage the rich, pretrained editing capabilities of text-to-image diffusion models by inverting visual prompts into editing instructions.
  arXiv  Detail & Related papers  (2023-07-26T17:50:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.