StreamingEffect: Real-Time Human-Centric Video Effect Generation

• URL: http://arxiv.org/abs/2605.17019v1
• Date: Sat, 16 May 2026 14:45:32 GMT
• Title: StreamingEffect: Real-Time Human-Centric Video Effect Generation
• Abstract summary: We present textbfStreamingEffect, a real-time human-centric streaming video effect framework.<n>Our method enables real-time, high-quality 720p video editing on a single H200 GPU.
• Score: 63.354447770285894
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Streaming video effect generation is highly desirable for live human-centric applications such as e-commerce streaming, entertainment, and vlogging, yet remains difficult due to the lack of suitable data and deployable editing models. Unlike generic video generation, this task requires real-time video-to-video editing that adds expressive effects while preserving human identity, background content, and temporal consistency. Existing acceleration efforts mainly focus on text-to-video generation, while efficient distillation for video editing remains largely underexplored. In this paper, we present \textbf{StreamingEffect}, a real-time human-centric streaming video effect framework. We adopt an in-context video editing architecture and train a high-quality bidirectional teacher, then distill it into a causal autoregressive student and further reduce sampling from 50 steps to 4 steps. We also introduce keyframe control, allowing reference effect frames to be injected online and propagated through the stream for interactive editing. To address the data bottleneck, we construct \textbf{VideoEffect-130K}, to our knowledge the largest human-centric video effect dataset, containing 70K effect videos and 60K editing videos across 600 effect categories curated from short-video and editing platforms. Experiments show that our method enables real-time, high-quality 720p video editing on a single H200 GPU.

Related papers

• EditaLive! Unified Character Video Editing for Live Streaming [59.93029261408041]
  EditaLive is a novel framework for real-time streaming character video editing.<n>It delivers state-of-the-art editing performance with faithful preservation of facial expressions and low-latency real-time streaming inference.
  arXiv  Detail & Related papers  (2026-08-27T13:37:45Z)
• JoyAI-Video-Edit: Real-Time Open-Ended Video Editing with Autoregressive Diffusion [85.58072988629806]
  JoyAI-Video-Edit is a 16B- parameter autoregressive diffusion framework for real-time, open-ended video editing.<n>Our method combines chunk-wise autoregressive adaptation, Source-Anchored Distribution Matching Distillation, and Long-Horizon Autoregressive Distillation.<n>JoyAI-Video-Edit substantially outperforms existing streaming editors and remains competitive with strong offline systems on both short and long videos.
  arXiv  Detail & Related papers  (2026-08-04T17:40:48Z)
• MotionV2V: Editing Motion in a Video [53.791975554391534]
  We propose modifying video motion by editing sparse trajectories extracted from the input.<n>We term the deviation between input and output trajectories a "motion edit"<n>Our approach allows for edits that start at any timestamp and propagate naturally.
  arXiv  Detail & Related papers  (2025-11-25T18:57:25Z)
• Señorita-2M: A High-Quality Instruction-based Dataset for General Video Editing by Video Specialists [17.451911831989293]
  We introduce Senorita-2M, a high-quality video editing dataset.<n>It is built by crafting four high-quality, specialized video editing models.<n>We propose a filtering pipeline to eliminate poorly edited video pairs.
  arXiv  Detail & Related papers  (2025-02-10T17:58:22Z)
• EffiVED:Efficient Video Editing via Text-instruction Diffusion Models [9.287394166165424]
  We introduce EffiVED, an efficient diffusion-based model that supports instruction-guided video editing. We transform vast image editing datasets and open-world videos into a high-quality dataset for training EffiVED.
  arXiv  Detail & Related papers  (2024-03-18T08:42:08Z)
• Neural Video Fields Editing [56.558490998753456]
  NVEdit is a text-driven video editing framework designed to mitigate memory overhead and improve consistency. We construct a neural video field, powered by tri-plane and sparse grid, to enable encoding long videos with hundreds of frames. Next, we update the video field through off-the-shelf Text-to-Image (T2I) models to text-driven editing effects.
  arXiv  Detail & Related papers  (2023-12-12T14:48:48Z)
• Hashing Neural Video Decomposition with Multiplicative Residuals in Space-Time [14.015909536844337]
  We present a video decomposition method that facilitates layer-based editing of videos withtemporally varying lighting effects. Our method efficiently learns layer-based neural representations of a 1080p video in 25s per frame via coordinate hashing. We propose to adopt evaluation metrics for objectively assessing the consistency of video editing.
  arXiv  Detail & Related papers  (2023-09-25T10:36:14Z)
• MagicEdit: High-Fidelity and Temporally Coherent Video Editing [70.55750617502696]
  We present MagicEdit, a surprisingly simple yet effective solution to the text-guided video editing task. We found that high-fidelity and temporally coherent video-to-video translation can be achieved by explicitly disentangling the learning of content, structure and motion signals during training.
  arXiv  Detail & Related papers  (2023-08-28T17:56:22Z)
• MagicVideo: Efficient Video Generation With Latent Diffusion Models [76.95903791630624]
  We present an efficient text-to-video generation framework based on latent diffusion models, termed MagicVideo. Due to a novel and efficient 3D U-Net design and modeling video distributions in a low-dimensional space, MagicVideo can synthesize video clips with 256x256 spatial resolution on a single GPU card. We conduct extensive experiments and demonstrate that MagicVideo can generate high-quality video clips with either realistic or imaginary content.
  arXiv  Detail & Related papers  (2022-11-20T16:40:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.