SEDiT: Mask-Free Video Subtitle Erasure via One-step Diffusion Transformer

• URL: http://arxiv.org/abs/2605.14894v1
• Date: Thu, 14 May 2026 14:37:54 GMT
• Title: SEDiT: Mask-Free Video Subtitle Erasure via One-step Diffusion Transformer
• Abstract summary: We present SEDiT, a novel one-stage video Subtitle Erasure approach via One-step Diffusion Transformer.<n>We introduce a mask-free inference approach that enables direct erasure of the targeted subtitle.<n>Thanks to one-step and chunk-wise streaming inference, our method can efficiently handle native 1440p video with infinite length.
• Score: 4.205766319033597
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent breakthroughs in video diffusion models have significantly accelerated the development of video editing techniques. However, existing methods often rely on inpainting video frames based on masked input, which requires extracting the target video mask in advance, and the precision of the segmentation directly affects the quality of the completion. In this paper, we present SEDiT, a novel one-stage video Subtitle Erasure approach via One-step Diffusion Transformer. We introduce a mask-free inference approach that enables direct erasure of the targeted subtitle. The proposed one-stage framework mitigates the sub-optimality inherent in the two-stage processing of prior models. Since subtitle removal is a localized editing task in which most pixels remain unchanged, the underlying distribution shift is minimal, making it well-suited to one-step generation under rectified flow. We empirically validate the reliability of one-step denoising and further provide a formal theoretical justification. Under the localized-editing structure of subtitle removal, the conditional optimal transport (OT) map and its induced rectified flow velocity field are Lipschitz continuous with respect to the latent variable, which underpins the theoretical feasibility of one-step sampling. To address the challenge of long-term temporal consistency, we adopt a hybrid training strategy by occasionally conditioning the model with a clean first-frame latent. This facilitates temporal continuity, allowing each segment during inference to leverage the output of its predecessor. To avoid visible seams caused by cropping and reinserting processed targets, particularly in scenarios involving substantial motion, we feed the original video directly into SEDiT. Thanks to one-step and chunk-wise streaming inference, our method can efficiently handle native 1440p video with infinite length.

Related papers

• From Draft to Draft-Free: One-Step Video Object Removal via Privileged Distillation and Fast Planting [52.368954064430945]
  From-Draft-to-Draft-Free (D2DF) is a framework that transforms coarse drafts into refined videos by multiple steps.<n>The denoising process for a single video takes only about 1 second.<n>Draft-conditioned and draft-free versions achieve state-of-the-art performance on multiple metrics.
  arXiv  Detail & Related papers  (2026-07-16T13:29:27Z)
• Accelerating Diffusion-based Video Editing via Heterogeneous Caching: Beyond Full Computing at Sampled Denoising Timestep [37.62908191585867]
  HetCache is a training-free diffusion acceleration framework for video-to-video (MV2V) generation and editing.<n>It reduces redundant attention operations while maintaining editing consistency and fidelity.<n> Experiments show that HetCache achieves a noticeable acceleration, including a 2.67$times$ latency speedup and FLOPs reduction over commonly used foundation models.
  arXiv  Detail & Related papers  (2026-03-25T12:53:31Z)
• SAMA: Factorized Semantic Anchoring and Motion Alignment for Instruction-Guided Video Editing [76.349958946335]
  We present SAMA (factorized Semantic Anchoring and Motion Alignment), a framework that factorizes video editing into semantic anchoring and motion modeling.<n>First, we introduce Semantic Anchoring, which establishes a reliable visual anchor by jointly predicting semantic tokens and video latents at sparse anchor frames.<n>Second, Motion Alignment pre-trains the same backbone on motion-centric video restoration pretext tasks.
  arXiv  Detail & Related papers  (2026-03-19T17:59:51Z)
• Streaming Autoregressive Video Generation via Diagonal Distillation [50.13573884115673]
  Autoregressive models offer a natural framework for sequential frame synthesis but require heavy computation to achieve high fidelity.<n>We propose Diagonal Distillation to exploit temporal information across both video chunks and denoising steps.<n>Our method generates a 5-second video in 2.61 seconds (up to 31 FPS), achieving a 277.3x speedup over the undistilled model.
  arXiv  Detail & Related papers  (2026-03-10T10:45:24Z)
• Learning Stochastic Bridges for Video Object Removal via Video-to-Video Translation [24.263367201734628]
  We reformulate video object removal as a video-to-video translation task via a bridge model.<n>Unlike noise-d methods, our framework establishes a direct path from the source video (with objects) to the target video (objects removed).<n>This bridge formulation effectively leverages the input video as a strong structural prior, guiding the model to perform precise removal while ensuring that the filled regions are logically consistent with the surrounding environment.
  arXiv  Detail & Related papers  (2026-01-17T14:22:14Z)
• Unified Video Editing with Temporal Reasoner [20.19759768002609]
  We propose VideoCoF, a novel Chain-of-Frames approach inspired by Chain-of-Thought reasoning.<n>VideoCoF enforces a see, reason, then edit" procedure by compelling the video diffusion model to first predict reasoning tokens.<n>We demonstrate that with a minimal data cost of only 50k video pairs, VideoCoF achieves state-of-the-art performance on VideoCoF-Bench.
  arXiv  Detail & Related papers  (2025-12-08T11:50:18Z)
• Mobius: Text to Seamless Looping Video Generation via Latent Shift [50.04534295458244]
  We present Mobius, a novel method to generate seamlessly looping videos from text descriptions directly without any user annotations.<n>Our method repurposes the pre-trained video latent diffusion model for generating looping videos from text prompts without any training.
  arXiv  Detail & Related papers  (2025-02-27T17:33:51Z)
• COVE: Unleashing the Diffusion Feature Correspondence for Consistent Video Editing [57.76170824395532]
  Video editing is an emerging task, in which most current methods adopt the pre-trained text-to-image (T2I) diffusion model to edit the source video.<n>We propose COrrespondence-guided Video Editing (COVE) to achieve high-quality and consistent video editing.<n>COVE can be seamlessly integrated into the pre-trained T2I diffusion model without the need for extra training or optimization.
  arXiv  Detail & Related papers  (2024-06-13T06:27:13Z)
• MAVIN: Multi-Action Video Generation with Diffusion Models via Transition Video Infilling [19.004339956475498]
  MAVIN is designed to generate transition videos that seamlessly connect two given videos, forming a cohesive integrated sequence. We introduce a new metric, CLIP-RS (CLIP Relative Smoothness), to evaluate temporal coherence and smoothness, complementing traditional quality-based metrics. Experimental results on horse and tiger scenarios demonstrate MAVIN's superior performance in generating smooth and coherent video transitions.
  arXiv  Detail & Related papers  (2024-05-28T09:46:09Z)
• Edit-A-Video: Single Video Editing with Object-Aware Consistency [49.43316939996227]
  We propose a video editing framework given only a pretrained TTI model and a single text, video> pair, which we term Edit-A-Video. The framework consists of two stages: (1) inflating the 2D model into the 3D model by appending temporal modules tuning and on the source video (2) inverting the source video into the noise and editing with target text prompt and attention map injection. We present extensive experimental results over various types of text and videos, and demonstrate the superiority of the proposed method compared to baselines in terms of background consistency, text alignment, and video editing quality.
  arXiv  Detail & Related papers  (2023-03-14T14:35:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.