Motion-Aware Reasoning from Speech to Mask Tracks: Runner-up Solution for the MeViS-Audio Track of the 8th LSVOS Challenge 2026

• URL: http://arxiv.org/abs/2608.22337v1
• Date: Sun, 23 Aug 2026 10:11:17 GMT
• Title: Motion-Aware Reasoning from Speech to Mask Tracks: Runner-up Solution for the MeViS-Audio Track of the 8th LSVOS Challenge 2026
• Abstract summary: Speech-guided referring video object segmentation aims to recover the mask tracks of objects specified by a spoken motion description.<n>We introduce Speech2MaskTrack, our approach for the MeViS-Audio track of the 8th LSVOS Challenge.
• Score: 18.40525632040563
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Speech-guided referring video object segmentation aims to recover the mask tracks of objects specified by a spoken motion description. Here, speech carries a linguistic instruction rather than acoustic evidence from a sounding object, so a solution must connect speech recognition, motion-centric temporal grounding, mask tracking, and explicit no-target handling. We introduce Speech2MaskTrack, our approach for the MeViS-Audio track of the 8th LSVOS Challenge. Speech2MaskTrack transcribes the spoken query and compiles it into structured constraints over category, count, direction, interaction role, and temporal phase. SAM3.1 enumerates multiple instance tracks, which TRACE ranks using complete-trajectory motion and relation evidence. A frozen lexical presence gate may suppress the ranked SAM3.1 base prediction. When the gate predicts that a target is present, an available full-expression-conditioned SaSaSa2VA track replaces the SAM3.1 mask. Only outputs that remain empty enter GPT-assisted recovery, which invokes SaSaSa2VA again under query- and mask-level verification. Speech2MaskTrack achieved second place in the official challenge ranking.

Related papers

• Competitive Memory Readout for Robust Video Object Segmentation: 2nd Place Technical Report for the MOSEv2 Track of the 8th LSVOS Challenge [51.15679610275694]
  We present our solution for the 8th Large-scale Video Object (LSVOS) Challenge at ECCV 2026.<n>The challenge evaluates robust video object segmentation under complex temporal dynamics.<n>Our method builds on SAM3 and focuses on its memory readout.
  arXiv  Detail & Related papers  (2026-08-22T18:09:51Z)
• Key-Frame Reasoning with SAM3: Third Place Solution for the MeViS-Text Track of the 8th LSVOS Challenge [15.751702557806595]
  This report presents a two-stage, training-free solution for the MeViS-Text track of the 8th LSVOS Challenge.<n>The task requires a model to localize and segment the object specified by a natural-language expression throughout a video.<n>Our method ranked third on the challenge test set, obtaining J&F, J, F, N-acc, T-acc, and Final scores of 0.761, 0.7367, 0.7852, 0.8333, 0.9755, and 0.856593, respectively.
  arXiv  Detail & Related papers  (2026-08-18T02:11:03Z)
• Multi-Agent Target-Existence Verification and Learned Mask Geometry Refinement: Winning Report of the MeViS-Text Track at the 8th LSVOS Challenge 2026 [0.6416429054645992]
  We present the first-place solution to the MeViS-Text track of the 8th Large-scale Video Object (LSVOS) Challenge 2026.<n> referring video object segmentation guided by written motion expressions, including deceptive no-target expressions.<n>Our pipeline, SSUPER, resolves each expression into a visual concept, generates full-video candidate masklets with SAM3.1, and selects target IDs.
  arXiv  Detail & Related papers  (2026-08-11T21:47:52Z)
• Agreement-Based Audio-Visual Segmentation:Champion Report for the MeViS-Audio Track in the 8th LSVOS Challenge [46.49526425182438]
  Qwen3-ASR first converts speech into text.<n>We select the track that has the highest average mask agreement with the other candidates.<n>A video-level classifier combines visual, audio-visual, and within-video query scores to decide whether any target is present.
  arXiv  Detail & Related papers  (2026-08-10T11:43:31Z)
• The 1st Winner for 5th PVUW MeViS-Text Challenge: Strong MLLMs Meet SAM3 for Referring Video Object Segmentation [65.24213788883016]
  This report presents our winning solution to the 5th PVUW MeViS-Text Challenge.<n>We build a fully training-free pipeline that combines strong multimodal large language models with SAM3.<n>Our method ranks first on the PVUW 2026 MeViS-Text test set, achieving a Final score of 0.909064 and a J&F score of 0.7897026.
  arXiv  Detail & Related papers  (2026-04-01T02:42:30Z)
• Perception Test 2025: Challenge Summary and a Unified VQA Extension [56.23039846339896]
  Third Perception Test challenge was organised as a full-day workshop alongside the IEEE/CVF International Conference on Computer Vision (ICCV) 2025.<n>Its primary goal is to benchmark state-of-the-art video models and measure the progress in multimodal perception.<n>We summarise the results from the main Perception Test challenge, detailing both the existing tasks as well as novel additions to the benchmark.
  arXiv  Detail & Related papers  (2026-01-09T20:02:21Z)
• EchoTrack: Auditory Referring Multi-Object Tracking for Autonomous Driving [64.58258341591929]
  Auditory Referring Multi-Object Tracking (AR-MOT) is a challenging problem in autonomous driving. We put forward EchoTrack, an end-to-end AR-MOT framework with dual-stream vision transformers. We establish the first set of large-scale AR-MOT benchmarks.
  arXiv  Detail & Related papers  (2024-02-28T12:50:16Z)
• Exploratory Evaluation of Speech Content Masking [7.012446339121189]
  We introduce a toy problem that explores an emerging type of privacy called "content masking" We evaluate a baseline masking technique based on modifying sequences of discrete phone representations (phone codes) We investigate three different masking locations and three types of masking strategies: noise substitution, word deletion, and phone sequence reversal.
  arXiv  Detail & Related papers  (2024-01-08T14:56:03Z)
• Tracking Anything in High Quality [63.63653185865726]
  HQTrack is a framework for High Quality Tracking anything in videos. It consists of a video multi-object segmenter (VMOS) and a mask refiner (MR)
  arXiv  Detail & Related papers  (2023-07-26T06:19:46Z)
• CHiME-6 Challenge:Tackling Multispeaker Speech Recognition for Unsegmented Recordings [87.37967358673252]
  We organize the 6th CHiME Speech Separation and Recognition Challenge (CHiME-6) The challenge revisits the previous CHiME-5 challenge and further considers the problem of distant multi-microphone conversational speech diarization and recognition. This paper provides a baseline description of the CHiME-6 challenge for both segmented multispeaker speech recognition and unsegmented multispeaker speech recognition.
  arXiv  Detail & Related papers  (2020-04-20T12:59:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.