Physics-Aware Video Instance Removal Benchmark

• URL: http://arxiv.org/abs/2604.05898v1
• Date: Tue, 07 Apr 2026 14:02:53 GMT
• Title: Physics-Aware Video Instance Removal Benchmark
• Abstract summary: Video Instance Removal (VIR) requires removing target objects while maintaining background integrity and physical consistency.<n>Current benchmarks primarily assess visual plausibility, often overlooking the physical causalities, such as lingering shadows.<n>We introduce the Physics-Aware Video Instance Removal benchmark, featuring 95 high-quality videos annotated with instance-accurate masks and removal prompts.
• Score: 23.60215680694382
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video Instance Removal (VIR) requires removing target objects while maintaining background integrity and physical consistency, such as specular reflections and illumination interactions. Despite advancements in text-guided editing, current benchmarks primarily assess visual plausibility, often overlooking the physical causalities, such as lingering shadows, triggered by object removal. We introduce the Physics-Aware Video Instance Removal (PVIR) benchmark, featuring 95 high-quality videos annotated with instance-accurate masks and removal prompts. PVIR is partitioned into Simple and Hard subsets, the latter explicitly targeting complex physical interactions. We evaluate four representative methods, PISCO-Removal, UniVideo, DiffuEraser, and CoCoCo, using a decoupled human evaluation protocol across three dimensions to isolate semantic, visual, and spatial failures: instruction following, rendering quality, and edit exclusivity. Our results show that PISCO-Removal and UniVideo achieve state-of-the-art performance, while DiffuEraser frequently introduces blurring artifacts and CoCoCo struggles significantly with instruction following. The persistent performance drop on the Hard subset highlights the ongoing challenge of recovering complex physical side effects.

Related papers

• BeyondMasks: Evaluating Causal and Physical Consistency in Video Object Removal [27.338971504674983]
  We introduce BeyondMasks, a paired benchmark for causally consistent video object removal.<n>The dataset spans diverse photometric, geometric, volumetric, and dynamic interactions.<n>We propose CORE, a structured vision language model based evaluation protocol.
  arXiv  Detail & Related papers  (2026-08-20T14:37:40Z)
• Articulation in Prime: Primitive-Based Articulated Object Understanding from a Single Casual Video [18.06685620402906]
  Retrieving 3D kinematics of articulated objects from monocular video is a fundamental challenge in computer vision.<n>We propose a category-agnostic optimization framework that treats articulated object understanding as a primitive-fitting problem.<n>We also propose the AiP-synth and AiP-real benchmarks, featuring significant camera motion and heavy occlusions, and outperform existing methods.
  arXiv  Detail & Related papers  (2026-05-18T16:52:41Z)
• PhyCo: Learning Controllable Physical Priors for Generative Motion [55.59209981836171]
  We present PhyCo, a framework that introduces continuous, interpretable, and physically grounded control into video generation.<n>Our approach integrates three key components: (i) a large-scale dataset of over 100K photorealistic simulation videos where friction, restitution, deformation, and force are systematically varied across diverse scenarios; (ii) physics-supervised fine-tuning of a pretrained diffusion model using a ControlNet conditioned on pixel-aligned physical property maps; and (iii) VLM-guided reward optimization, where a fine-tuned vision-language model evaluates generated videos with targeted physics queries and provides differentiable feedback.
  arXiv  Detail & Related papers  (2026-04-30T17:53:03Z)
• From Understanding to Erasing: Towards Complete and Stable Video Object Removal [8.035439287403983]
  Video object removal aims to eliminate target objects from videos while plausibly completing missing regions and preservingtemporal consistency.<n>It remains challenging to remove object-induced side effects without compromising overall coherence.<n>This paper introduces understanding into erasing from two complementary perspectives.
  arXiv  Detail & Related papers  (2026-04-02T06:50:39Z)
• EffectErase: Joint Video Object Removal and Insertion for High-Quality Effect Erasing [50.43992550991499]
  Video object removal aims to eliminate dynamic target objects and their visual effects, such as deformation, shadows, and reflections, while restoring seamless backgrounds.<n>Recent diffusion-based video inpainting and object removal methods can remove the objects but often struggle to erase these effects and to synthesize coherent backgrounds.<n>We introduce VOR (Video Object Removal), a large-scale dataset that provides diverse paired videos.<n>We propose EffectErase, an effect-aware video object removal method that treats video object insertion as the inverse auxiliary task within a reciprocal learning scheme.
  arXiv  Detail & Related papers  (2026-03-19T17:59:22Z)
• Vectorized Video Representation with Easy Editing via Hierarchical Spatio-Temporally Consistent Proxy Embedding [45.593989778240655]
  A proposed representation achieves high video reconstruction accuracy with fewer parameters.<n>It supports complex video processing tasks, including video in-painting and temporally consistent video editing.
  arXiv  Detail & Related papers  (2025-10-14T08:05:30Z)
• MultiPhys: Multi-Person Physics-aware 3D Motion Estimation [28.91813849219037]
  We introduce MultiPhys, a method designed for recovering multi-person motion from monocular videos. Our focus lies in capturing coherent spatial placement between pairs of individuals across varying degrees of engagement. We devise a pipeline in which the motion estimated by a kinematic-based method is fed into a physics simulator in an autoregressive manner.
  arXiv  Detail & Related papers  (2024-04-18T08:29:29Z)
• Appearance-Based Refinement for Object-Centric Motion Segmentation [85.2426540999329]
  We introduce an appearance-based refinement method that leverages temporal consistency in video streams to correct inaccurate flow-based proposals. Our approach involves a sequence-level selection mechanism that identifies accurate flow-predicted masks as exemplars. Its performance is evaluated on multiple video segmentation benchmarks, including DAVIS, YouTube, SegTrackv2, and FBMS-59.
  arXiv  Detail & Related papers  (2023-12-18T18:59:51Z)
• Weakly-Supervised Video Object Grounding via Causal Intervention [82.68192973503119]
  We target at the task of weakly-supervised video object grounding (WSVOG), where only video-sentence annotations are available during model learning. It aims to localize objects described in the sentence to visual regions in the video, which is a fundamental capability needed in pattern analysis and machine learning.
  arXiv  Detail & Related papers  (2021-12-01T13:13:03Z)
• Weakly Supervised Human-Object Interaction Detection in Video via Contrastive Spatiotemporal Regions [81.88294320397826]
  A system does not know what human-object interactions are present in a video as or the actual location of the human and object. We introduce a dataset comprising over 6.5k videos with human-object interaction that have been curated from sentence captions. We demonstrate improved performance over weakly supervised baselines adapted to our annotations on our video dataset.
  arXiv  Detail & Related papers  (2021-10-07T15:30:18Z)
• Occlusion-Aware Video Object Inpainting [72.38919601150175]
  This paper presents occlusion-aware video object inpainting, which recovers both the complete shape and appearance for occluded objects in videos. Our technical contribution VOIN jointly performs video object shape completion and occluded texture generation. For more realistic results, VOIN is optimized using both T-PatchGAN and a newoc-temporal YouTube attention-based multi-class discriminator.
  arXiv  Detail & Related papers  (2021-08-15T15:46:57Z)
• Grounding Physical Concepts of Objects and Events Through Dynamic Visual Reasoning [84.90458333884443]
  We present the Dynamic Concept Learner (DCL), a unified framework that grounds physical objects and events from video and language. DCL can detect and associate objects across the frames, ground visual properties, and physical events, understand the causal relationship between events, make future and counterfactual predictions, and leverage these presentations for answering queries. DCL achieves state-of-the-art performance on CLEVRER, a challenging causal video reasoning dataset, even without using ground-truth attributes and collision labels from simulations for training.
  arXiv  Detail & Related papers  (2021-03-30T17:59:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.