Occlusion-Aware Physics-Semantic Keyframe Selection for Robust Video Editing

• URL: http://arxiv.org/abs/2605.23192v2
• Date: Wed, 27 May 2026 07:47:23 GMT
• Title: Occlusion-Aware Physics-Semantic Keyframe Selection for Robust Video Editing
• Abstract summary: We propose an occlusion-aware selection framework that automatically identifies an optimal anchor frame for downstream editing.<n>Our framework enables precise and temporally consistent editing without requiring manual annotations.
• Score: 53.06890499054828
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video editing has recently achieved remarkable progress with diffusion-based generative models, enabling diverse object-level manipulations from natural language instructions. However, existing methods often struggle under occlusion, viewpoint changes, and fast object motion, where unreliable visual observations lead to inaccurate localization, temporal flickering, and inconsistent edits. In this work, we identify the absence of reliable visual anchors as a fundamental bottleneck in occlusion-robust video editing. To address this issue, we propose an occlusion-aware physics-semantic keyframe selection framework that automatically identifies an optimal anchor frame for downstream editing. Specifically, our method evaluates candidate frames from three complementary perspectives: structural completeness for avoiding truncated observations, cycle-consistent tracking stability for measuring physical reliability, and vision-language-based attribute visibility for ensuring semantic clarity. The selected keyframe is then propagated through bidirectional tracking to generate dense spatiotemporal masks, which are used as auxiliary supervision for a diffusion-based video editing backbone. By transforming occlusion handling from explicit reconstruction into reliable anchor selection, our framework enables precise and temporally consistent editing without requiring manual annotations. Extensive experiments on challenging video editing benchmarks demonstrate the effectiveness and high-quality performance of our method.

Related papers

• Beyond Consistency: Preserving Temporal Structure in Zero-Shot Video Editing [48.768011768488584]
  Existing zero-shot video editing methods fail to preserve the video's original temporal structure.<n>We introduce a novel zero-shot editing approach that explicitly focuses on preserving the source video's temporal structure.<n>Our method achieves state-of-the-art results, successfully balancing the preservation of original temporal structure with computational efficiency.
  arXiv  Detail & Related papers  (2026-06-07T18:53:57Z)
• FlowAnchor: Stabilizing the Editing Signal for Inversion-Free Video Editing [7.08737643046839]
  FlowAnchor is a training-free framework for stable and efficient inversion-free, flow-based video editing.<n>It anchors both where to edit and how strongly to edit in high-dimensional video latent spaces.<n>It achieves more faithful, temporally coherent, and computationally efficient video editing across challenging multi-object and fast-motion scenarios.
  arXiv  Detail & Related papers  (2026-04-24T14:17:11Z)
• SAMA: Factorized Semantic Anchoring and Motion Alignment for Instruction-Guided Video Editing [76.349958946335]
  We present SAMA (factorized Semantic Anchoring and Motion Alignment), a framework that factorizes video editing into semantic anchoring and motion modeling.<n>First, we introduce Semantic Anchoring, which establishes a reliable visual anchor by jointly predicting semantic tokens and video latents at sparse anchor frames.<n>Second, Motion Alignment pre-trains the same backbone on motion-centric video restoration pretext tasks.
  arXiv  Detail & Related papers  (2026-03-19T17:59:51Z)
• Zero-Shot Video Translation and Editing with Frame Spatial-Temporal Correspondence [81.82643953694485]
  We present FRESCO, which integrates intra-frame correspondence with inter-frame correspondence to formulate a more robust spatial-temporal constraint.<n>Our method goes beyond attention guidance to explicitly optimize features, achieving high spatial-temporal consistency with the input video.<n>We verify FRESCO adaptations on two zero-shot tasks of video-to-video translation and text-guided video editing.
  arXiv  Detail & Related papers  (2025-12-03T15:51:11Z)
• VALA: Learning Latent Anchors for Training-Free and Temporally Consistent [29.516179213427694]
  We propose VALA, a variational alignment module that adaptively selects key frames and compresses their latent features into semantic anchors for consistent video editing.<n>Our method can be fully integrated into training-free text-to-image based video editing models.
  arXiv  Detail & Related papers  (2025-10-27T03:44:11Z)
• Motion-Aware Concept Alignment for Consistent Video Editing [57.08108545219043]
  We introduce MoCA-Video (Motion-Aware Concept Alignment in Video), a training-free framework bridging the gap between image-domain semantic mixing and video.<n>Given a generated video and a user-provided reference image, MoCA-Video injects the semantic features of the reference image into a specific object within the video.<n>We evaluate MoCA's performance using the standard SSIM, image-level LPIPS, temporal LPIPS, and introduce a novel metric CASS (Conceptual Alignment Shift Score) to evaluate the consistency and effectiveness of the visual shifts between the source prompt and the modified video frames
  arXiv  Detail & Related papers  (2025-06-01T13:28:04Z)
• SSVOD: Semi-Supervised Video Object Detection with Sparse Annotations [12.139451002212063]
  SSVOD exploits motion dynamics of videos to utilize large-scale unlabeled frames with sparse annotations. Our method achieves significant performance improvements over existing methods on ImageNet-VID, Epic-KITCHENS, and YouTube-VIS.
  arXiv  Detail & Related papers  (2023-09-04T06:41:33Z)
• RIGID: Recurrent GAN Inversion and Editing of Real Face Videos [73.97520691413006]
  GAN inversion is indispensable for applying the powerful editability of GAN to real images. Existing methods invert video frames individually often leading to undesired inconsistent results over time. We propose a unified recurrent framework, named textbfRecurrent vtextbfIdeo textbfGAN textbfInversion and etextbfDiting (RIGID) Our framework learns the inherent coherence between input frames in an end-to-end manner.
  arXiv  Detail & Related papers  (2023-08-11T12:17:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.