Learning Stochastic Bridges for Video Object Removal via Video-to-Video Translation

• URL: http://arxiv.org/abs/2601.12066v1
• Date: Sat, 17 Jan 2026 14:22:14 GMT
• Title: Learning Stochastic Bridges for Video Object Removal via Video-to-Video Translation
• Abstract summary: We reformulate video object removal as a video-to-video translation task via a bridge model.<n>Unlike noise-d methods, our framework establishes a direct path from the source video (with objects) to the target video (objects removed).<n>This bridge formulation effectively leverages the input video as a strong structural prior, guiding the model to perform precise removal while ensuring that the filled regions are logically consistent with the surrounding environment.
• Score: 24.263367201734628
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing video object removal methods predominantly rely on diffusion models following a noise-to-data paradigm, where generation starts from uninformative Gaussian noise. This approach discards the rich structural and contextual priors present in the original input video. Consequently, such methods often lack sufficient guidance, leading to incomplete object erasure or the synthesis of implausible content that conflicts with the scene's physical logic. In this paper, we reformulate video object removal as a video-to-video translation task via a stochastic bridge model. Unlike noise-initialized methods, our framework establishes a direct stochastic path from the source video (with objects) to the target video (objects removed). This bridge formulation effectively leverages the input video as a strong structural prior, guiding the model to perform precise removal while ensuring that the filled regions are logically consistent with the surrounding environment. To address the trade-off where strong bridge priors hinder the removal of large objects, we propose a novel adaptive mask modulation strategy. This mechanism dynamically modulates input embeddings based on mask characteristics, balancing background fidelity with generative flexibility. Extensive experiments demonstrate that our approach significantly outperforms existing methods in both visual quality and temporal consistency.

Related papers

• SEDiT: Mask-Free Video Subtitle Erasure via One-step Diffusion Transformer [4.205766319033597]
  We present SEDiT, a novel one-stage video Subtitle Erasure approach via One-step Diffusion Transformer.<n>We introduce a mask-free inference approach that enables direct erasure of the targeted subtitle.<n>Thanks to one-step and chunk-wise streaming inference, our method can efficiently handle native 1440p video with infinite length.
  arXiv  Detail & Related papers  (2026-05-14T14:37:54Z)
• VOID: Video Object and Interaction Deletion [50.155301620235285]
  Existing video object removal methods excel at inpainting content "behind" the object and correcting appearance-level artifacts such as shadows and reflections.<n>We present VOID, a video object removal framework designed to perform physically-plausible inpainting in complex scenarios.
  arXiv  Detail & Related papers  (2026-04-02T17:36:53Z)
• From Understanding to Erasing: Towards Complete and Stable Video Object Removal [8.035439287403983]
  Video object removal aims to eliminate target objects from videos while plausibly completing missing regions and preservingtemporal consistency.<n>It remains challenging to remove object-induced side effects without compromising overall coherence.<n>This paper introduces understanding into erasing from two complementary perspectives.
  arXiv  Detail & Related papers  (2026-04-02T06:50:39Z)
• Unified Video Editing with Temporal Reasoner [20.19759768002609]
  We propose VideoCoF, a novel Chain-of-Frames approach inspired by Chain-of-Thought reasoning.<n>VideoCoF enforces a see, reason, then edit" procedure by compelling the video diffusion model to first predict reasoning tokens.<n>We demonstrate that with a minimal data cost of only 50k video pairs, VideoCoF achieves state-of-the-art performance on VideoCoF-Bench.
  arXiv  Detail & Related papers  (2025-12-08T11:50:18Z)
• Beyond Flicker: Detecting Kinematic Inconsistencies for Generalizable Deepfake Video Detection [41.44337153700898]
  Generalizing deepfake detection to unseen manipulations remains a key challenge.<n>Recent approach is to train a network with pristine face images that have been manipulated with hand-crafted artifacts to extract more generalizable clues.<n>We propose a synthetic video generation method that creates training data with subtle inconsistencies.
  arXiv  Detail & Related papers  (2025-12-03T19:00:07Z)
• Generative Omnimatte: Learning to Decompose Video into Layers [29.098471541412113]
  We present a novel generative layered video decomposition framework to address the omnimatte problem.<n>Our core idea is to train a video diffusion model to identify and remove scene effects caused by a specific object.<n>We show that this model can be finetuned from an existing video inpainting model with a small, carefully curated dataset.
  arXiv  Detail & Related papers  (2024-11-25T18:59:57Z)
• DiffUHaul: A Training-Free Method for Object Dragging in Images [78.93531472479202]
  We propose a training-free method, dubbed DiffUHaul, for the object dragging task. We first apply attention masking in each denoising step to make the generation more disentangled across different objects. In the early denoising steps, we interpolate the attention features between source and target images to smoothly fuse new layouts with the original appearance.
  arXiv  Detail & Related papers  (2024-06-03T17:59:53Z)
• Motion-Zero: Zero-Shot Moving Object Control Framework for Diffusion-Based Video Generation [10.5019872575418]
  We propose a novel zero-shot moving object trajectory control framework, Motion-Zero, to enable a bounding-box-trajectories-controlled text-to-video diffusion model.<n>Our method can be flexibly applied to various state-of-the-art video diffusion models without any training process.
  arXiv  Detail & Related papers  (2024-01-18T17:22:37Z)
• Appearance-Based Refinement for Object-Centric Motion Segmentation [85.2426540999329]
  We introduce an appearance-based refinement method that leverages temporal consistency in video streams to correct inaccurate flow-based proposals. Our approach involves a sequence-level selection mechanism that identifies accurate flow-predicted masks as exemplars. Its performance is evaluated on multiple video segmentation benchmarks, including DAVIS, YouTube, SegTrackv2, and FBMS-59.
  arXiv  Detail & Related papers  (2023-12-18T18:59:51Z)
• Style-A-Video: Agile Diffusion for Arbitrary Text-based Video Style Transfer [13.098901971644656]
  This paper proposes a zero-shot video stylization method named Style-A-Video. Uses a generative pre-trained transformer with an image latent diffusion model to achieve a concise text-controlled video stylization. Tests show that we can attain superior content preservation and stylistic performance while incurring less consumption than previous solutions.
  arXiv  Detail & Related papers  (2023-05-09T14:03:27Z)
• Video Demoireing with Relation-Based Temporal Consistency [68.20281109859998]
  Moire patterns, appearing as color distortions, severely degrade image and video qualities when filming a screen with digital cameras. We study how to remove such undesirable moire patterns in videos, namely video demoireing.
  arXiv  Detail & Related papers  (2022-04-06T17:45:38Z)
• VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modeling [88.30109041658618]
  A great challenge in video-language (VidL) modeling lies in the disconnection between fixed video representations extracted from image/video understanding models and downstream VidL data. We present VIOLET, a fully end-to-end VIdeO-LanguagE Transformer, which adopts a video transformer to explicitly model the temporal dynamics of video inputs.
  arXiv  Detail & Related papers  (2021-11-24T18:31:20Z)
• Video Salient Object Detection via Contrastive Features and Attention Modules [106.33219760012048]
  We propose a network with attention modules to learn contrastive features for video salient object detection. A co-attention formulation is utilized to combine the low-level and high-level features. We show that the proposed method requires less computation, and performs favorably against the state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-11-03T17:40:32Z)
• TransMoMo: Invariance-Driven Unsupervised Video Motion Retargeting [107.39743751292028]
  TransMoMo is capable of transferring motion of a person in a source video realistically to another video of a target person. We exploit invariance properties of three factors of variation including motion, structure, and view-angle. We demonstrate the effectiveness of our method over the state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-03-31T17:49:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.