JoyAI-Video-Edit: Real-Time Open-Ended Video Editing with Autoregressive Diffusion

• URL: http://arxiv.org/abs/2608.03974v1
• Date: Tue, 04 Aug 2026 17:40:48 GMT
• Title: JoyAI-Video-Edit: Real-Time Open-Ended Video Editing with Autoregressive Diffusion
• Abstract summary: JoyAI-Video-Edit is a 16B- parameter autoregressive diffusion framework for real-time, open-ended video editing.<n>Our method combines chunk-wise autoregressive adaptation, Source-Anchored Distribution Matching Distillation, and Long-Horizon Autoregressive Distillation.<n>JoyAI-Video-Edit substantially outperforms existing streaming editors and remains competitive with strong offline systems on both short and long videos.
• Score: 85.58072988629806
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Real-time video editing requires low-latency causal generation with bounded computational resources while preserving source fidelity and long-term temporal consistency. We present JoyAI-Video-Edit, a 16B-parameter autoregressive diffusion framework for real-time, open-ended video editing without access to future frames or a predefined video duration. Our method combines chunk-wise autoregressive adaptation, Source-Anchored Distribution Matching Distillation (SA-DMD), and Long-Horizon Autoregressive Distillation to reduce train--inference mismatch, preserve source fidelity during two-step generation, and mitigate accumulated temporal drift. Extensive automatic and human evaluations show that JoyAI-Video-Edit substantially outperforms existing streaming editors and remains competitive with strong offline systems on both short and long videos. The complete system achieves end-to-end 720p video editing at approximately 30 FPS on a single Nvidia B200 GPU. Code is available at https://github.com/jd-opensource/JoyAI-Video-Edit.

Related papers

• Dynamic Video Generation: Shaping Video Generation Across Time and Space [9.269520155517094]
  We propose DVG, a framework that automatically selects content-aware acceleration strategies without manual tuning or retraining.<n> DVG achieves near-lossless acceleration across models and tasks, reaching up to 7 times speedup on HunyuanVideo and HunyuanVideo-1.51.5, and 18 times when combined with distillation.
  arXiv  Detail & Related papers  (2026-05-20T11:24:02Z)
• StreamingEffect: Real-Time Human-Centric Video Effect Generation [63.354447770285894]
  We present textbfStreamingEffect, a real-time human-centric streaming video effect framework.<n>Our method enables real-time, high-quality 720p video editing on a single H200 GPU.
  arXiv  Detail & Related papers  (2026-05-16T14:45:32Z)
• Latent Inter-Frame Pruning: A Training-Free Method Bridging Traditional Video Compression and Modern Diffusion Transformers for Efficient Generation [2.639628765667673]
  Video generation, while capable of generating realistic videos, is computationally expensive and slow, prohibiting real-time applications.<n>We observe that video latents encoded via an autoencoder contain redundancy along the temporal axis.<n>We propose the Latent Inter-frame Pruning framework to prune (skip the re-computation of) duplicated latent patches, thereby reducing computational burden and increasing throughput.
  arXiv  Detail & Related papers  (2026-04-26T20:03:11Z)
• Adaptive Caching for Faster Video Generation with Diffusion Transformers [52.73348147077075]
  Diffusion Transformers (DiTs) rely on larger models and heavier attention mechanisms, resulting in slower inference speeds. We introduce a training-free method to accelerate video DiTs, termed Adaptive Caching (AdaCache) We also introduce a Motion Regularization (MoReg) scheme to utilize video information within AdaCache, controlling the compute allocation based on motion content.
  arXiv  Detail & Related papers  (2024-11-04T18:59:44Z)
• Video-Infinity: Distributed Long Video Generation [73.30145218077074]
  Diffusion models have recently achieved remarkable results for video generation. Our method generates videos up to 2,300 frames in approximately 5 minutes, enabling long video generation at a speed 100 times faster than the prior methods.
  arXiv  Detail & Related papers  (2024-06-24T01:56:12Z)
• StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text [58.49820807662246]
  We introduce StreamingT2V, an autoregressive approach for long video generation of 80, 240, 600, 1200 or more frames with smooth transitions.<n>Our code will be available at: https://github.com/Picsart-AI-Research/StreamingT2V.
  arXiv  Detail & Related papers  (2024-03-21T18:27:29Z)
• LOVECon: Text-driven Training-Free Long Video Editing with ControlNet [9.762680144118061]
  This paper aims to bridge the gap, establishing a simple and effective baseline for training-free diffusion model-based long video editing. We build the pipeline upon ControlNet, which excels at various image editing tasks based on text prompts. Our method manages to edit videos comprising hundreds of frames according to user requirements.
  arXiv  Detail & Related papers  (2023-10-15T02:39:25Z)
• Ground-A-Video: Zero-shot Grounded Video Editing using Text-to-image Diffusion Models [65.268245109828]
  Ground-A-Video is a video-to-video translation framework for multi-attribute video editing. It attains temporally consistent editing of input videos in a training-free manner. Experiments and applications demonstrate that Ground-A-Video's zero-shot capacity outperforms other baseline methods in terms of edit-accuracy and frame consistency.
  arXiv  Detail & Related papers  (2023-10-02T11:28:37Z)
• VidEdit: Zero-Shot and Spatially Aware Text-Driven Video Editing [18.24307442582304]
  We introduce VidEdit, a novel method for zero-shot text-based video editing. Our experiments show that VidEdit outperforms state-of-the-art methods on DAVIS dataset.
  arXiv  Detail & Related papers  (2023-06-14T19:15:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.