VideoSEG-O3: A Multi-turn Reinforcement Learning Framework for Reasoning Video Object Segmentation

• URL: http://arxiv.org/abs/2606.06819v1
• Date: Fri, 05 Jun 2026 01:44:38 GMT
• Title: VideoSEG-O3: A Multi-turn Reinforcement Learning Framework for Reasoning Video Object Segmentation
• Abstract summary: Reasoning Video Object (RVOS) demands a sophisticated integration of temporal dynamics, spatial details, and linguistic reasoning.<n>We propose textbfVideoSEG-O3, the first multi-turn reinforcement learning framework for RVOS.
• Score: 38.77426504855051
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reasoning Video Object Segmentation (RVOS) demands a sophisticated integration of temporal dynamics, spatial details, and linguistic reasoning to achieve precise pixel-level localization. Existing methods are limited to reasoning over fixed initial inputs and lack the capacity to actively acquire further visual evidence, which is often essential for resolving complex references in long or intricate videos. To address this, we propose \textbf{VideoSEG-O3}, the first multi-turn reinforcement learning framework for RVOS that emulates the human \textit{``coarse-to-fine''} cognitive process. It employs a \textit{multi-turn temporal-spatial chain-of-thought} to capture fine-grained details by iteratively pinpointing critical intervals and keyframes. Additionally, to enable the policy to perceive segmentation quality beyond mere text probability of \texttt{[SEG]} during the RL stage, we introduce \textit{SEG-aware logit calibration}, which integrates pixel-wise segmentation feedback directly into the token-level logits. Furthermore, we design a \textit{decoupled thinking trace} to hierarchically decompose the reasoning process into temporal, spatial, and linguistic dimensions, and construct \textbf{VTS-CoT}, a specialized cold-start dataset featuring comprehensive reasoning trajectories. The code and models will be released at https://github.com/Dmmm1997/VideoSEG-O3.

Related papers

• Smart-Insertion-V: Photorealistic Video Insertion via a Closed-Loop Feedback Dual-Stream Framework [52.477407591581176]
  Mask-free video object insertion has emerged as a challenging task, requiring harmonious integration of reference objects into source videos.<n>We propose textittextbfSmart-Insertion-V, an end-to-end textbfDual-Stream framework that concurrently conducts video insertion and image style transfer.
  arXiv  Detail & Related papers  (2026-05-22T17:54:54Z)
• ToLL: Topological Layout Learning with Structural Multi-view Augmentation for 3D Scene Graph Pretraining [29.799719863247287]
  3D Scene Graph (3DSG) generation plays a pivotal role in spatial understanding and semantic-affordance perception.<n>Current solutions primarily focus on cross-modal assisted representation learning and object-centric generation pre-training.<n>We propose a Topological Layout Learning (ToLL) for 3DSG pretraining framework.<n>In detail, we design an Anchor-Conditioned Topological Geometry Reasoning, with a GNN to recover the global layout of zero-centered subgraphs by the spatial priors from sparse anchors.
  arXiv  Detail & Related papers  (2026-03-30T08:43:52Z)
• VideoAtlas: Navigating Long-Form Video in Logarithmic Compute [3.705718227493618]
  textbfVideoAtlas is a task-agnostic environment to represent video as a hierarchical grid.<n>The hierarchical structure ensures access depth grows only logarithmically with video length.<n>Video-RLM remains the most duration-robust method with minimal accuracy degradation when scaling from 1-hour to 10-hour benchmarks.
  arXiv  Detail & Related papers  (2026-03-18T17:20:19Z)
• Video Spatial Reasoning with Object-Centric 3D Rollout [58.12446467377404]
  We propose Object-Centric 3D Rollout (OCR) to enable robust video spatial reasoning.<n>OCR introduces structured perturbations to the 3D geometry of selected objects during training.<n>OCR compels the model to reason holistically across the entire scene.
  arXiv  Detail & Related papers  (2025-11-17T09:53:41Z)
• Xiaoice: Training-Free Video Understanding via Self-Supervised Spatio-Temporal Clustering of Semantic Features [10.21556794551883]
  This paper introduces a novel, training-free framework for video understanding that circumvents end-to-end training.<n>Our core idea is to video understanding as a self-supervised-temporal clustering within a high-dimensional feature space.<n>This approach provides an effective, interpretable, and model-agnostic pathway for zero-shot, automated structural analysis of video content.
  arXiv  Detail & Related papers  (2025-10-19T10:13:34Z)
• MomentSeg: Moment-Centric Sampling for Enhanced Video Pixel Understanding [40.37010049965347]
  Referring Video Object (RefVOS) seeks to segment target objects in videos guided by natural language descriptions.<n>We propose a unified framework that jointly optimize Temporal Sentence Grounding (TSG) and RefVOS, naturally incorporating key moment grounding capability.
  arXiv  Detail & Related papers  (2025-10-10T11:18:21Z)
• Temporal Prompting Matters: Rethinking Referring Video Object Segmentation [64.82333675385802]
  Referring Video Object (RVOS) aims to segment the object referred to by the query sentence in the video.<n>Most existing methods require end-to-end training with dense mask annotations.<n>We propose a Temporal Prompt Generation and Selection (Tenet) framework to address the referring and video factors.
  arXiv  Detail & Related papers  (2025-10-08T17:59:57Z)
• Deforming Videos to Masks: Flow Matching for Referring Video Segmentation [46.416906762916305]
  FlowRVS is a novel framework that reconceptualizes RVOS as a conditional continuous flow problem.<n>We reformulate the task by learning a direct, language-guided deformation from a video's holistic representation to its target mask.<n>Our one-stage, generative approach achieves new state-of-the-art results across all major RVOS benchmarks.
  arXiv  Detail & Related papers  (2025-10-07T17:14:10Z)
• Unleashing Hierarchical Reasoning: An LLM-Driven Framework for Training-Free Referring Video Object Segmentation [17.238084264485988]
  Referring Video Object (RVOS) aims to segment an object of interest throughout a video based on a language description.<n>bftextPARSE-VOS is a training-free framework powered by Large Language Models (LLMs)<n>bftextPARSE-VOS achieved state-of-the-art performance on three major benchmarks: Ref-YouTube-VOS, Ref-DAVIS17, and MeViS.
  arXiv  Detail & Related papers  (2025-09-06T15:46:23Z)
• Hierarchical Temporal Context Learning for Camera-based Semantic Scene Completion [57.232688209606515]
  We present HTCL, a novel Temporal Temporal Context Learning paradigm for improving camera-based semantic scene completion. Our method ranks $1st$ on the Semantic KITTI benchmark and even surpasses LiDAR-based methods in terms of mIoU.
  arXiv  Detail & Related papers  (2024-07-02T09:11:17Z)
• Constructing Holistic Spatio-Temporal Scene Graph for Video Semantic Role Labeling [96.64607294592062]
  Video Semantic Label Roleing (VidSRL) aims to detect salient events from given videos. Recent endeavors have put forth methods for VidSRL, but they can be subject to two key drawbacks.
  arXiv  Detail & Related papers  (2023-08-09T17:20:14Z)
• Spatial-Temporal Correlation and Topology Learning for Person Re-Identification in Videos [78.45050529204701]
  We propose a novel framework to pursue discriminative and robust representation by modeling cross-scale spatial-temporal correlation. CTL utilizes a CNN backbone and a key-points estimator to extract semantic local features from human body. It explores a context-reinforced topology to construct multi-scale graphs by considering both global contextual information and physical connections of human body.
  arXiv  Detail & Related papers  (2021-04-15T14:32:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.