Robust Promptable Video Object Segmentation

• URL: http://arxiv.org/abs/2605.12006v1
• Date: Tue, 12 May 2026 11:55:31 GMT
• Title: Robust Promptable Video Object Segmentation
• Abstract summary: This paper offers the first comprehensive study on robust PVOS (RobustPVOS)<n>We first construct a new, comprehensive benchmark with two real-world evaluation datasets of 351 video clips and more than 2,500 object masks.<n>We present a new RobustPVOS method, dubbed Memory-object-conditioned Gated-rank Adaptation (MoGA)
• Score: 67.1533741758339
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The performance of promptable video object segmentation (PVOS) models substantially degrades under input corruptions, which prevents PVOS deployment in safety-critical domains. This paper offers the first comprehensive study on robust PVOS (RobustPVOS). We first construct a new, comprehensive benchmark with two real-world evaluation datasets of 351 video clips and more than 2,500 object masks under real-world adverse conditions. At the same time, we generate synthetic training data by applying diverse and temporally varying corruptions to existing VOS datasets. Moreover, we present a new RobustPVOS method, dubbed Memory-object-conditioned Gated-rank Adaptation (MoGA). The key to successfully performing RobustPVOS is two-fold: effectively handling object-specific degradation and ensuring temporal consistency in predictions. MoGA leverages object-specific representations maintained in memory across frames to condition the robustification process, which allows the model to handle each tracked object differently in a temporally consistent way. Extensive experiments on our benchmark validate MoGA's efficacy, showing consistent and significant improvements across diverse corruption types on both synthetic and real-world datasets, establishing a strong baseline for future RobustPVOS research. Our benchmark is publicly available at https://sohyun-l.github.io/RobustPVOS_project_page/.

Related papers

• VIPER: Process-aware Evaluation for Generative Video Reasoning [64.86465792516658]
  We introduce VIPER, a comprehensive benchmark spanning 16 tasks across temporal, structural, symbolic, spatial, physics, and planning reasoning.<n>Our experiments reveal that state-of-the-art video models achieve only about 20% POC@1.0 and exhibit a significant outcome-hacking.
  arXiv  Detail & Related papers  (2025-12-31T16:31:59Z)
• RoHOI: Robustness Benchmark for Human-Object Interaction Detection [84.78366452133514]
  Human-Object Interaction (HOI) detection is crucial for robot-human assistance, enabling context-aware support.<n>We introduce the first benchmark for HOI detection, evaluating model resilience under diverse challenges.<n>Our benchmark, RoHOI, includes 20 corruption types based on the HICO-DET and V-COCO datasets and a new robustness-focused metric.
  arXiv  Detail & Related papers  (2025-07-12T01:58:04Z)
• CronusVLA: Towards Efficient and Robust Manipulation via Multi-Frame Vision-Language-Action Modeling [84.51372201195132]
  CronusVLA is a unified framework that extends single-frame VLA models to the multi-frame paradigm.<n>CronusVLA achieves leading performance and superior robustness, with a 70.9% success rate.<n>These results highlight the potential of efficient multi-frame adaptation in VLA models for more powerful and robust real-world deployment.
  arXiv  Detail & Related papers  (2025-06-24T17:30:27Z)
• Prior2Former -- Evidential Modeling of Mask Transformers for Assumption-Free Open-World Panoptic Segmentation [74.55677741919035]
  We propose Prior2Former (P2F), the first approach for segmentation vision transformers rooted in evidential learning.<n>P2F extends the mask vision transformer architecture by incorporating a Beta prior for computing model uncertainty in pixel-wise binary mask assignments.<n>Unlike most segmentation models addressing unknown classes, P2F operates without access to OOD data samples or contrastive training on void (i.e., unlabeled) classes.
  arXiv  Detail & Related papers  (2025-04-07T08:53:14Z)
• ReferDINO: Referring Video Object Segmentation with Visual Grounding Foundations [33.74746234704817]
  Video object segmentation (RVOS) aims to segment target objects throughout a video based on a text description.<n>This is challenging as it involves deep vision-level understanding, pixel-level dense prediction andtemporal reasoning.<n>We propose bfReferDINO RVOS that inherits region-level vision-text alignment from foundational visual grounding models.
  arXiv  Detail & Related papers  (2025-01-24T16:24:15Z)
• OneVOS: Unifying Video Object Segmentation with All-in-One Transformer Framework [24.947436083365925]
  OneVOS is a novel framework that unifies the core components of VOS with All-in-One Transformer. OneVOS achieves state-of-the-art performance across 7 datasets, particularly excelling in complex LVOS and MOSE datasets with 70.1% and 66.4% $J & F$, surpassing previous state-of-the-art methods by 4.2% and 7.0%, respectively.
  arXiv  Detail & Related papers  (2024-03-13T16:38:26Z)
• Video Object Segmentation in Panoptic Wild Scenes [31.701108453289162]
  We introduce semi-supervised video object segmentation (VOS) to panoptic wild scenes. We present a large-scale benchmark as well as a baseline method for it. Experimental results show that VIPOSeg can not only boost the performance of VOS models by panoptic training but also evaluate them comprehensively in panoptic scenes.
  arXiv  Detail & Related papers  (2023-05-08T05:46:59Z)
• Robust Object Detection via Instance-Level Temporal Cycle Confusion [89.1027433760578]
  We study the effectiveness of auxiliary self-supervised tasks to improve the out-of-distribution generalization of object detectors. Inspired by the principle of maximum entropy, we introduce a novel self-supervised task, instance-level temporal cycle confusion (CycConf) For each object, the task is to find the most different object proposals in the adjacent frame in a video and then cycle back to itself for self-supervision.
  arXiv  Detail & Related papers  (2021-04-16T21:35:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.