Online Reasoning Video Object Segmentation

• URL: http://arxiv.org/abs/2604.11411v1
• Date: Mon, 13 Apr 2026 12:55:56 GMT
• Title: Online Reasoning Video Object Segmentation
• Abstract summary: Reasoning video object segmentation predicts pixel-level masks in videos from natural-language queries.<n>We study Online Reasoning Video Object (ORVOS), where models incrementally interpret queries using only past and current frames.<n>We introduce ORVOSB, a benchmark with frame-level causal annotations and referent-shift labels, comprising 210 videos, 12,907 annotated frames, and 512 queries across five reasoning categories.
• Score: 26.75863941739301
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reasoning video object segmentation predicts pixel-level masks in videos from natural-language queries that may involve implicit and temporally grounded references. However, existing methods are developed and evaluated in an offline regime, where the entire video is available at inference time and future frames can be exploited for retrospective disambiguation, deviating from real-world deployments that require strictly causal, frame-by-frame decisions. We study Online Reasoning Video Object Segmentation (ORVOS), where models must incrementally interpret queries using only past and current frames without revisiting previous predictions, while handling referent shifts as events unfold. To support evaluation, we introduce ORVOSB, a benchmark with frame-level causal annotations and referent-shift labels, comprising 210 videos, 12,907 annotated frames, and 512 queries across five reasoning categories. We further propose a baseline with continually-updated segmentation prompts and a structured temporal token reservoir for long-horizon reasoning under bounded computation. Experiments show that existing methods struggle under strict causality and referent shifts, while our baseline establishes a strong foundation for future research.

Related papers

• REVEAL: A Rubric-Guided Agent for Explicit Evidence Sufficiency Verificationin Long-Video Question Answering [25.519927761448603]
  REVEAL is a rubric-guided agent framework for long-video question answering.<n>We introduce an adaptive visual-similarity-based preprocessing pipeline that groups visually coherent adjacent frames into natural event units.<n>Built upon this structured memory, REVEAL uses an automatically constructed rubric library to explicitly verify whether retrieved evidence satisfies sufficiency criteria.
  arXiv  Detail & Related papers  (2026-08-09T09:55:42Z)
• Explainable Forensics of Manipulated Segments in Untrimmed Long Videos [50.190474724159465]
  We formulate the task of Temporal AI-Generated Segment localization and Explanation.<n>We introduce TASLE, a large-scale benchmark comprising 12,472 untrimmed videos with diverse manipulation patterns and rich annotation signals.<n>We propose MSLoc, a coarse-to-fine forensic baseline that combines a boundary-sensitive proposal generation module for efficient long-video scanning with an MLLM-based refinement module for precise boundary localization and interpretable reasoning.
  arXiv  Detail & Related papers  (2026-06-01T15:48:38Z)
• ClipTBP: Clip-Pair based Temporal Boundary Prediction with Boundary-Aware Learning for Moment Retrieval [56.989259812432]
  We propose ClipTBP, a clip-pair temporal boundary prediction framework based on boundary-aware learning.<n>We show that ClipTBP consistently improves performance when applied to various existing models and demonstrates more robust boundary prediction performance even in ambiguous query scenarios.
  arXiv  Detail & Related papers  (2026-04-30T08:41:41Z)
• VIRST: Video-Instructed Reasoning Assistant for SpatioTemporal Segmentation [6.447274127678917]
  Video Video Object (RVOS) aims to segment target objects in videos based on natural language descriptions.<n>We propose VIRST (Video-Instructed Reasoning Assistant for Spatio-Temporal), an end-to-end framework that unifies global video reasoning and pixel-level mask prediction within a single model.
  arXiv  Detail & Related papers  (2026-03-28T00:34:15Z)
• Temporal Prompting Matters: Rethinking Referring Video Object Segmentation [64.82333675385802]
  Referring Video Object (RVOS) aims to segment the object referred to by the query sentence in the video.<n>Most existing methods require end-to-end training with dense mask annotations.<n>We propose a Temporal Prompt Generation and Selection (Tenet) framework to address the referring and video factors.
  arXiv  Detail & Related papers  (2025-10-08T17:59:57Z)
• Temporally-Constrained Video Reasoning Segmentation and Automated Benchmark Construction [8.214041057237491]
  We introduce temporally-constrained video reasoning segmentation, a novel task formulation that requires models to implicitly infer when target objects become contextually relevant.<n>We also present TCVideoRS, a temporally-constrained video RS dataset containing 52 samples using the videos from the MVOR dataset.
  arXiv  Detail & Related papers  (2025-07-22T15:59:21Z)
• Logic-in-Frames: Dynamic Keyframe Search via Visual Semantic-Logical Verification for Long Video Understanding [23.022070084937603]
  We introduce a semantics-driven search framework that reformulates selection under the paradigm of Visual Semantic-Logical Search.<n>Our method establishes new SOTA performance on the manually annotated benchmark in key-frame selection metrics.
  arXiv  Detail & Related papers  (2025-03-17T13:07:34Z)
• Temporally Consistent Referring Video Object Segmentation with Hybrid Memory [98.80249255577304]
  We propose an end-to-end R-VOS paradigm that explicitly models temporal consistency alongside the referring segmentation. Features of frames with automatically generated high-quality reference masks are propagated to segment remaining frames. Extensive experiments demonstrate that our approach enhances temporal consistency by a significant margin.
  arXiv  Detail & Related papers  (2024-03-28T13:32:49Z)
• Appearance-Based Refinement for Object-Centric Motion Segmentation [85.2426540999329]
  We introduce an appearance-based refinement method that leverages temporal consistency in video streams to correct inaccurate flow-based proposals. Our approach involves a sequence-level selection mechanism that identifies accurate flow-predicted masks as exemplars. Its performance is evaluated on multiple video segmentation benchmarks, including DAVIS, YouTube, SegTrackv2, and FBMS-59.
  arXiv  Detail & Related papers  (2023-12-18T18:59:51Z)
• Temporal-aware Hierarchical Mask Classification for Video Semantic Segmentation [62.275143240798236]
  Video semantic segmentation dataset has limited categories per video. Less than 10% of queries could be matched to receive meaningful gradient updates during VSS training. Our method achieves state-of-the-art performance on the latest challenging VSS benchmark VSPW without bells and whistles.
  arXiv  Detail & Related papers  (2023-09-14T20:31:06Z)
• OnlineRefer: A Simple Online Baseline for Referring Video Object Segmentation [75.07460026246582]
  Referring video object segmentation (RVOS) aims at segmenting an object in a video following human instruction. Current state-of-the-art methods fall into an offline pattern, in which each clip independently interacts with text embedding. We propose a simple yet effective online model using explicit query propagation, named OnlineRefer.
  arXiv  Detail & Related papers  (2023-07-18T15:43:35Z)
• Boundary-sensitive Pre-training for Temporal Localization in Videos [124.40788524169668]
  We investigate model pre-training for temporal localization by introducing a novel boundary-sensitive pretext ( BSP) task. With the synthesized boundaries, BSP can be simply conducted via classifying the boundary types. Extensive experiments show that the proposed BSP is superior and complementary to the existing action classification based pre-training counterpart.
  arXiv  Detail & Related papers  (2020-11-21T17:46:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.