Key-Frame Reasoning with SAM3: Third Place Solution for the MeViS-Text Track of the 8th LSVOS Challenge

• URL: http://arxiv.org/abs/2608.17279v1
• Date: Tue, 18 Aug 2026 02:11:03 GMT
• Title: Key-Frame Reasoning with SAM3: Third Place Solution for the MeViS-Text Track of the 8th LSVOS Challenge
• Abstract summary: This report presents a two-stage, training-free solution for the MeViS-Text track of the 8th LSVOS Challenge.<n>The task requires a model to localize and segment the object specified by a natural-language expression throughout a video.<n>Our method ranked third on the challenge test set, obtaining J&F, J, F, N-acc, T-acc, and Final scores of 0.761, 0.7367, 0.7852, 0.8333, 0.9755, and 0.856593, respectively.
• Score: 15.751702557806595
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This report presents a two-stage, training-free solution for the MeViS-Text track of the 8th LSVOS Challenge. The task requires a model to localize and segment the object specified by a natural-language expression throughout a video. Such expressions often depend on temporal cues, including actions, interactions, directions, and relative positions. Our first stage uses Gemini-3.1 Pro via API to decompose a video-level event into instance-level targets, select a key frame for each target, and generate a discriminative description aligned with that frame. In the second stage, SAM3-agent produces a pixel-level seed mask on the selected frame, and the SAM3 video tracker propagates the mask bidirectionally through the video. Valid instances are grounded and propagated independently before their frame-wise masks are merged. All local SAM3 processing runs on a single NVIDIA GeForce RTX 4090 without task-specific training or model ensembling. Our method ranked third on the challenge test set, obtaining J&F, J, F, N-acc., T-acc., and Final scores of 0.761, 0.7367, 0.7852, 0.8333, 0.9755, and 0.856593, respectively.

Related papers

• Report of the 8th LSVOS Challenge: Complex and Multimodal Video Object Segmentation [123.08518673784818]
  This report summarizes the 8th Large-scale Video Object (LSVOS) Challenge, held in conjunction with ECCV 2026.<n>The challenge evaluates video segmentation in three complementary settings: complex semi-supervised video object segmentation on MOSEv2, text-guided referring video object segmentation on MeViSv2-Text, and audio-guided referring video object segmentation on MeViSv2-Audio.
  arXiv  Detail & Related papers  (2026-09-05T13:11:38Z)
• Agreement-Based Audio-Visual Segmentation:Champion Report for the MeViS-Audio Track in the 8th LSVOS Challenge [46.49526425182438]
  Qwen3-ASR first converts speech into text.<n>We select the track that has the highest average mask agreement with the other candidates.<n>A video-level classifier combines visual, audio-visual, and within-video query scores to decide whether any target is present.
  arXiv  Detail & Related papers  (2026-08-10T11:43:31Z)
• The 1st Winner for 5th PVUW MeViS-Text Challenge: Strong MLLMs Meet SAM3 for Referring Video Object Segmentation [65.24213788883016]
  This report presents our winning solution to the 5th PVUW MeViS-Text Challenge.<n>We build a fully training-free pipeline that combines strong multimodal large language models with SAM3.<n>Our method ranks first on the PVUW 2026 MeViS-Text test set, achieving a Final score of 0.909064 and a J&F score of 0.7897026.
  arXiv  Detail & Related papers  (2026-04-01T02:42:30Z)
• SegEarth-OV3: Exploring SAM 3 for Open-Vocabulary Semantic Segmentation in Remote Sensing Images [51.42466259821335]
  We present a preliminary exploration of applying SAM 3 to the remote sensing OVSS task without any training.<n>First, we implement a mask fusion strategy that combines the outputs from SAM 3's semantic segmentation head and the Transformer decoder.<n>Second, we utilize the presence score from the presence head to filter out categories that do not exist in the scene.
  arXiv  Detail & Related papers  (2025-12-09T15:42:28Z)
• Evaluating SAM2 for Video Semantic Segmentation [60.157605818225186]
  The Anything Model 2 (SAM2) has proven to be a powerful foundation model for promptable visual object segmentation in both images and videos.<n>This paper explores the extension of SAM2 to dense Video Semantic (VSS)<n>Our experiments suggest that leveraging SAM2 enhances overall performance in VSS, primarily due to its precise predictions of object boundaries.
  arXiv  Detail & Related papers  (2025-12-01T15:15:16Z)
• SAMITE: Position Prompted SAM2 with Calibrated Memory for Visual Object Tracking [58.35852822355312]
  Visual Object Tracking (VOT) is widely used in applications like autonomous driving to continuously track targets in videos.<n>To address these issues, some methods propose to adapt the video foundation model SAM2 for VOT, where the tracking results of each frame would be encoded as memory for conditioning the rest of frames in an autoregressive manner.<n>We present a SAMITE model, built upon SAM2 with additional modules, to tackle these challenges.
  arXiv  Detail & Related papers  (2025-07-29T12:11:56Z)
• The Instance-centric Transformer for the RVOS Track of LSVOS Challenge: 3rd Place Solution [9.282159019651433]
  We build two instance-centric models and fuse predicted results from frame-level and instance-level. Our method achieved a score of 52.67 J&F in the validation phase and 60.36 J&F in the test phase, securing the final ranking of 3rd place in the 6-th LSVOS Challenge RVOS Track.
  arXiv  Detail & Related papers  (2024-08-20T04:45:13Z)
• SAM-PD: How Far Can SAM Take Us in Tracking and Segmenting Anything in Videos by Prompt Denoising [37.216493829454706]
  We explore the potential of applying the Segment Anything Model to track and segment objects in videos. Specifically, we iteratively propagate the bounding box of each object's mask in the preceding frame as the prompt for the next frame. To enhance SAM's denoising capability against position and size variations, we propose a multi-prompt strategy.
  arXiv  Detail & Related papers  (2024-03-07T03:52:59Z)
• SAMPro3D: Locating SAM Prompts in 3D for Zero-Shot Instance Segmentation [24.733049281032272]
  We introduce SAMPro3D for zero-shot instance segmentation of 3D scenes.<n>Our approach segments 3D instances by applying the pretrained Segment Anything Model (SAM) to 2D frames.<n>Our method achieves comparable or better performance compared to previous zero-shot or fully supervised approaches.
  arXiv  Detail & Related papers  (2023-11-29T15:11:03Z)
• Segment Anything Meets Point Tracking [116.44931239508578]
  This paper presents a novel method for point-centric interactive video segmentation, empowered by SAM and long-term point tracking. We highlight the merits of point-based tracking through direct evaluation on the zero-shot open-world Unidentified Video Objects (UVO) benchmark. Our experiments on popular video object segmentation and multi-object segmentation tracking benchmarks, including DAVIS, YouTube-VOS, and BDD100K, suggest that a point-based segmentation tracker yields better zero-shot performance and efficient interactions.
  arXiv  Detail & Related papers  (2023-07-03T17:58:01Z)
• 3rd Place Solution for PVUW2023 VSS Track: A Large Model for Semantic Segmentation on VSPW [68.56017675820897]
  In this paper, we introduce 3rd place solution for PVUW2023 VSS track. We have explored various image-level visual backbones and segmentation heads to tackle the problem of video semantic segmentation.
  arXiv  Detail & Related papers  (2023-06-04T07:50:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.