The 2nd Solution for LSVOS Challenge RVOS Track: Spatial-temporal Refinement for Consistent Semantic Segmentation

• URL: http://arxiv.org/abs/2408.12447v1
• Date: Thu, 22 Aug 2024 14:43:02 GMT
• Title: The 2nd Solution for LSVOS Challenge RVOS Track: Spatial-temporal Refinement for Consistent Semantic Segmentation
• Authors: Tuyen Tran,
• Abstract summary: We propose a method to enhance the temporal consistency of the referring object segmentation model. Our method placed 2nd in the final ranking of the RVOS Track at the ECCV 2024 LSVOS Challenge.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Referring Video Object Segmentation (RVOS) is a challenging task due to its requirement for temporal understanding. Due to the obstacle of computational complexity, many state-of-the-art models are trained on short time intervals. During testing, while these models can effectively process information over short time steps, they struggle to maintain consistent perception over prolonged time sequences, leading to inconsistencies in the resulting semantic segmentation masks. To address this challenge, we take a step further in this work by leveraging the tracking capabilities of the newly introduced Segment Anything Model version 2 (SAM-v2) to enhance the temporal consistency of the referring object segmentation model. Our method achieved a score of 60.40 \mathcal{J\text{\&}F} on the test set of the MeViS dataset, placing 2nd place in the final ranking of the RVOS Track at the ECCV 2024 LSVOS Challenge.

Related papers

• Solution for OOD-CV UNICORN Challenge 2024 Object Detection Assistance LLM Counting Ability Improvement [6.621745547882088]
  This report provides a detailed description of the method that we explored and proposed in the ECCV OOD-CV UNICORN Challenge 2024. The dataset of this competition are OODCA-VQA and SketchyQA. Our approach ranked second in the final test with a score of 0.86.
  arXiv  Detail & Related papers  (2024-10-05T15:11:47Z)
• Discriminative Spatial-Semantic VOS Solution: 1st Place Solution for 6th LSVOS [68.47681139026666]
  Video object segmentation (VOS) is a crucial task in computer vision. Current VOS methods struggle with complex scenes and prolonged object motions. This report introduces a discriminative spatial-temporal VOS model.
  arXiv  Detail & Related papers  (2024-08-29T10:47:17Z)
• DeTra: A Unified Model for Object Detection and Trajectory Forecasting [68.85128937305697]
  Our approach formulates the union of the two tasks as a trajectory refinement problem. To tackle this unified task, we design a refinement transformer that infers the presence, pose, and multi-modal future behaviors of objects. In our experiments, we observe that ourmodel outperforms the state-of-the-art on Argoverse 2 Sensor and Open dataset.
  arXiv  Detail & Related papers  (2024-06-06T18:12:04Z)
• A novel dual-stream time-frequency contrastive pretext tasks framework for sleep stage classification [1.9399172852087767]
  This study introduces a dual-stream pretext task architecture that operates both in the time and frequency domains. We have examined the incorporation of the novel Frequency Similarity (FS) pretext task into two existing pretext tasks, Relative Positioning (RP) and Temporal Shuffling (TS) The inclusion of FS resulted in a notable improvement in downstream task accuracy, with a 1.28 percent improvement on RP and a 2.02 percent improvement on TS.
  arXiv  Detail & Related papers  (2023-12-15T09:05:06Z)
• Temporal-aware Hierarchical Mask Classification for Video Semantic Segmentation [62.275143240798236]
  Video semantic segmentation dataset has limited categories per video. Less than 10% of queries could be matched to receive meaningful gradient updates during VSS training. Our method achieves state-of-the-art performance on the latest challenging VSS benchmark VSPW without bells and whistles.
  arXiv  Detail & Related papers  (2023-09-14T20:31:06Z)
• Leveraging Foundation models for Unsupervised Audio-Visual Segmentation [49.94366155560371]
  Audio-Visual (AVS) aims to precisely outline audible objects in a visual scene at the pixel level. Existing AVS methods require fine-grained annotations of audio-mask pairs in supervised learning fashion. We introduce unsupervised audio-visual segmentation with no need for task-specific data annotations and model training.
  arXiv  Detail & Related papers  (2023-09-13T05:05:47Z)
• 1st Place Solution for YouTubeVOS Challenge 2021:Video Instance Segmentation [0.39146761527401414]
  Video Instance (VIS) is a multi-task problem performing detection, segmentation, and tracking simultaneously. We propose two modules, named Temporally Correlated Instance (TCIS) and Bidirectional Tracking (BiTrack) By combining these techniques with a bag of tricks, the network performance is significantly boosted compared to the baseline.
  arXiv  Detail & Related papers  (2021-06-12T00:20:38Z)
• STEP: Segmenting and Tracking Every Pixel [107.23184053133636]
  We present a new benchmark: Segmenting and Tracking Every Pixel (STEP) Our work is the first that targets this task in a real-world setting that requires dense interpretation in both spatial and temporal domains. For measuring the performance, we propose a novel evaluation metric and Tracking Quality (STQ)
  arXiv  Detail & Related papers  (2021-02-23T18:43:02Z)
• Fast Video Object Segmentation With Temporal Aggregation Network and Dynamic Template Matching [67.02962970820505]
  We introduce "tracking-by-detection" into Video Object (VOS) We propose a new temporal aggregation network and a novel dynamic time-evolving template matching mechanism to achieve significantly improved performance. We achieve new state-of-the-art performance on the DAVIS benchmark without complicated bells and whistles in both speed and accuracy, with a speed of 0.14 second per frame and J&F measure of 75.9% respectively.
  arXiv  Detail & Related papers  (2020-07-11T05:44:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.