The 1st Winner for 5th PVUW MeViS-Text Challenge: Strong MLLMs Meet SAM3 for Referring Video Object Segmentation

• URL: http://arxiv.org/abs/2604.00404v1
• Date: Wed, 01 Apr 2026 02:42:30 GMT
• Title: The 1st Winner for 5th PVUW MeViS-Text Challenge: Strong MLLMs Meet SAM3 for Referring Video Object Segmentation
• Abstract summary: This report presents our winning solution to the 5th PVUW MeViS-Text Challenge.<n>We build a fully training-free pipeline that combines strong multimodal large language models with SAM3.<n>Our method ranks first on the PVUW 2026 MeViS-Text test set, achieving a Final score of 0.909064 and a J&F score of 0.7897026.
• Score: 65.24213788883016
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This report presents our winning solution to the 5th PVUW MeViS-Text Challenge. The track studies referring video object segmentation under motion-centric language expressions, where the model must jointly understand appearance, temporal behavior, and object interactions. To address this problem, we build a fully training-free pipeline that combines strong multimodal large language models with SAM3. Our method contains three stages. First, Gemini-3.1 Pro decomposes each target event into instance-level grounding targets, selects the frame where the target is most clearly visible, and generates a discriminative description. Second, SAM3-agent produces a precise seed mask on the selected frame, and the official SAM3 tracker propagates the mask through the whole video. Third, a refinement stage uses Qwen3.5-Plus and behavior-level verification to correct ambiguous or semantically inconsistent predictions. Without task-specific fine-tuning, our method ranks first on the PVUW 2026 MeViS-Text test set, achieving a Final score of 0.909064 and a J&F score of 0.7897. The code is available at https://github.com/Moujuruo/MeViSv2_Track_Solution_2026.

Related papers

• Key-Frame Reasoning with SAM3: Third Place Solution for the MeViS-Text Track of the 8th LSVOS Challenge [15.751702557806595]
  This report presents a two-stage, training-free solution for the MeViS-Text track of the 8th LSVOS Challenge.<n>The task requires a model to localize and segment the object specified by a natural-language expression throughout a video.<n>Our method ranked third on the challenge test set, obtaining J&F, J, F, N-acc, T-acc, and Final scores of 0.761, 0.7367, 0.7852, 0.8333, 0.9755, and 0.856593, respectively.
  arXiv  Detail & Related papers  (2026-08-18T02:11:03Z)
• Multi-Agent Target-Existence Verification and Learned Mask Geometry Refinement: Winning Report of the MeViS-Text Track at the 8th LSVOS Challenge 2026 [0.6416429054645992]
  We present the first-place solution to the MeViS-Text track of the 8th Large-scale Video Object (LSVOS) Challenge 2026.<n> referring video object segmentation guided by written motion expressions, including deceptive no-target expressions.<n>Our pipeline, SSUPER, resolves each expression into a visual concept, generates full-video candidate masklets with SAM3.1, and selects target IDs.
  arXiv  Detail & Related papers  (2026-08-11T21:47:52Z)
• Agreement-Based Audio-Visual Segmentation:Champion Report for the MeViS-Audio Track in the 8th LSVOS Challenge [46.49526425182438]
  Qwen3-ASR first converts speech into text.<n>We select the track that has the highest average mask agreement with the other candidates.<n>A video-level classifier combines visual, audio-visual, and within-video query scores to decide whether any target is present.
  arXiv  Detail & Related papers  (2026-08-10T11:43:31Z)
• Zero-Shot Object Re-Identification in Egocentric Kitchen Videos via Multi-Stage SAM3 Feature Fusion [59.32099847726022]
  We study zero-shot object ReID on the EPIC-Kitchens benchmark.<n>The goal is to match active food and kitchen-tool instances across frames using only pre-trained visual features.
  arXiv  Detail & Related papers  (2026-05-25T23:01:41Z)
• Advancing Complex Video Object Segmentation via Tracking-Enhanced Prompt: The 1st Winner for 5th PVUW MOSE Challenge [65.24213788883016]
  In the Complex Video Object task, researchers are required to track and segment specific targets within cluttered environments.<n>The root cause of this limitation lies in SAM3's insufficient comprehension of these specific target types.<n>We propose TEP: Advancing Complex Video Object via Tracking-Enhanced Prompts.
  arXiv  Detail & Related papers  (2026-04-01T02:23:23Z)
• Evaluating SAM2 for Video Semantic Segmentation [60.157605818225186]
  The Anything Model 2 (SAM2) has proven to be a powerful foundation model for promptable visual object segmentation in both images and videos.<n>This paper explores the extension of SAM2 to dense Video Semantic (VSS)<n>Our experiments suggest that leveraging SAM2 enhances overall performance in VSS, primarily due to its precise predictions of object boundaries.
  arXiv  Detail & Related papers  (2025-12-01T15:15:16Z)
• SimToken: A Simple Baseline for Referring Audio-Visual Segmentation [29.88252418748085]
  Referring Audio-Visual (Ref-AVS) aims to segment specific objects in videos based on natural language expressions.<n>This task poses significant challenges in cross-modal reasoning and fine-grained object localization.<n>We propose a framework, SimToken, that integrates a multimodal large language model (MLLM) with the Segment Anything Model (SAM)
  arXiv  Detail & Related papers  (2025-09-22T08:55:04Z)
• GeoSAM2: Unleashing the Power of SAM2 for 3D Part Segmentation [81.0871900167463]
  We introduce GeoSAM2, a prompt-controllable framework for 3D part segmentation.<n>Given a textureless object, we render normal and point maps from predefined viewpoints.<n>We accept simple 2D prompts - clicks or boxes - to guide part selection.<n>The predicted masks are back-projected to the object and aggregated across views.
  arXiv  Detail & Related papers  (2025-08-19T17:58:51Z)
• Decoupled Seg Tokens Make Stronger Reasoning Video Segmenter and Grounder [5.57393627015653]
  Video segmenter and grounder approaches, exemplified by Sa2VA, directly fuse features within segmentation models.<n>This often results in an undesirable entanglement of dynamic visual information and static semantics, thereby degrading segmentation accuracy.<n>We propose DeSa2VA, a decoupling-enhanced prompting scheme integrating text pre-training and a linear decoupling module to address the information processing limitations inherent in SAM-2.
  arXiv  Detail & Related papers  (2025-06-28T13:30:36Z)
• 4th PVUW MeViS 3rd Place Report: Sa2VA [105.88675577642204]
  We show that with a simple modification to the test time inference method on stronger MLLMs, we can lead to stronger results on MeVIS.<n>In particular, we adopt the recent method Sa2VA, a unified model for dense grounded understanding of both images and videos.
  arXiv  Detail & Related papers  (2025-04-01T07:06:47Z)
• 1st Place Solution for 5th LSVOS Challenge: Referring Video Object Segmentation [65.45702890457046]
  We integrate strengths of leading RVOS models to build up an effective paradigm. To improve the consistency and quality of masks, we propose Two-Stage Multi-Model Fusion strategy. Our method achieves 75.7% J&F on Ref-Youtube-VOS validation set and 70% J&F on test set, which ranks 1st place on 5th Large-scale Video Object Challenge (ICCV 2023) track 3.
  arXiv  Detail & Related papers  (2024-01-01T04:24:48Z)
• Fully Transformer-Equipped Architecture for End-to-End Referring Video Object Segmentation [24.814534011440877]
  We propose an end-to-end RVOS framework which treats the RVOS task as a mask sequence learning problem. To capture the object-level spatial context, we have developed the Stacked Transformer. The model finds the best matching between mask sequence and text query.
  arXiv  Detail & Related papers  (2023-09-21T09:47:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.