The Instance-centric Transformer for the RVOS Track of LSVOS Challenge: 3rd Place Solution

• URL: http://arxiv.org/abs/2408.10541v1
• Date: Tue, 20 Aug 2024 04:45:13 GMT
• Title: The Instance-centric Transformer for the RVOS Track of LSVOS Challenge: 3rd Place Solution
• Abstract summary: We build two instance-centric models and fuse predicted results from frame-level and instance-level. Our method achieved a score of 52.67 J&F in the validation phase and 60.36 J&F in the test phase, securing the final ranking of 3rd place in the 6-th LSVOS Challenge RVOS Track.
• Score: 9.282159019651433
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Referring Video Object Segmentation is an emerging multi-modal task that aims to segment objects in the video given a natural language expression. In this work, we build two instance-centric models and fuse predicted results from frame-level and instance-level. First, we introduce instance mask into the DETR-based model for query initialization to achieve temporal enhancement and employ SAM for spatial refinement. Secondly, we build an instance retrieval model conducting binary instance mask classification whether the instance is referred. Finally, we fuse predicted results and our method achieved a score of 52.67 J&F in the validation phase and 60.36 J&F in the test phase, securing the final ranking of 3rd place in the 6-th LSVOS Challenge RVOS Track.

Related papers

• SOS! : A Streamlined Object-Conditional Transformer for Model-free Segmentation [67.361660179434]
  We propose SOS (Streamlined Object-conditional Transformer for model-free pipelines) for unseen objects segmentation.<n>Central to our framework is a novel Object-Conditional Transformer that learns identity-anchored queries, unifying mask generation and target identification into a single feed-forward pass.<n>Sos establishes a new state-of-the-art for model-free unseen objects segmentation, delivering accurate and high-efficiency performance.
  arXiv  Detail & Related papers  (2026-08-15T15:53:23Z)
• Evaluating SAM2 for Video Semantic Segmentation [60.157605818225186]
  The Anything Model 2 (SAM2) has proven to be a powerful foundation model for promptable visual object segmentation in both images and videos.<n>This paper explores the extension of SAM2 to dense Video Semantic (VSS)<n>Our experiments suggest that leveraging SAM2 enhances overall performance in VSS, primarily due to its precise predictions of object boundaries.
  arXiv  Detail & Related papers  (2025-12-01T15:15:16Z)
• Beyond the Final Layer: Hierarchical Query Fusion Transformer with Agent-Interpolation Initialization for 3D Instance Segmentation [33.58208166717537]
  3D instance segmentation aims to predict a set of object instances in a scene and represent them as binary foreground masks with corresponding semantic labels. transformer-based methods are gaining increasing attention due to their elegant pipelines, reduced manual selection of geometric properties, and superior performance.
  arXiv  Detail & Related papers  (2025-02-06T15:19:48Z)
• Adapting Segment Anything Model for Unseen Object Instance Segmentation [70.60171342436092]
  Unseen Object Instance (UOIS) is crucial for autonomous robots operating in unstructured environments. We propose UOIS-SAM, a data-efficient solution for the UOIS task. UOIS-SAM integrates two key components: (i) a Heatmap-based Prompt Generator (HPG) to generate class-agnostic point prompts with precise foreground prediction, and (ii) a Hierarchical Discrimination Network (HDNet) that adapts SAM's mask decoder.
  arXiv  Detail & Related papers  (2024-09-23T19:05:50Z)
• Adapting Pre-Trained Vision Models for Novel Instance Detection and Segmentation [15.414518995812754]
  We propose a unified framework (NIDS-Net) comprising object proposal generation, embedding creation for both instance templates and proposal regions, and embedding matching for instance label assignment. Our framework surpasses current state-of-the-art methods, demonstrating notable improvements of 22.3, 46.2, 10.3, and 24.0 in average precision (AP) across four detection datasets.
  arXiv  Detail & Related papers  (2024-05-28T06:16:57Z)
• 1st Place Solution for 5th LSVOS Challenge: Referring Video Object Segmentation [65.45702890457046]
  We integrate strengths of leading RVOS models to build up an effective paradigm. To improve the consistency and quality of masks, we propose Two-Stage Multi-Model Fusion strategy. Our method achieves 75.7% J&F on Ref-Youtube-VOS validation set and 70% J&F on test set, which ranks 1st place on 5th Large-scale Video Object Challenge (ICCV 2023) track 3.
  arXiv  Detail & Related papers  (2024-01-01T04:24:48Z)
• EipFormer: Emphasizing Instance Positions in 3D Instance Segmentation [51.996943482875366]
  We present a novel Transformer-based architecture, EipFormer, which comprises progressive aggregation and dual position embedding. EipFormer achieves superior or comparable performance compared to state-of-the-art approaches.
  arXiv  Detail & Related papers  (2023-12-09T16:08:47Z)
• Mask4Former: Mask Transformer for 4D Panoptic Segmentation [13.99703660936949]
  Mask4Former is the first transformer-based approach unifying semantic instance segmentation and tracking. Our model directly predicts semantic instances their temporal associations without relying on hand-crafted non-learned association strategies. Mask4Former achieves a new state-of-the-art on the SemanticTITI test set with a score of 68.4 LSTQ.
  arXiv  Detail & Related papers  (2023-09-28T03:30:50Z)
• Mask3D: Mask Transformer for 3D Semantic Instance Segmentation [89.41640045953378]
  We show that we can leverage generic Transformer building blocks to directly predict instance masks from 3D point clouds. Using Transformer decoders, the instance queries are learned by iteratively attending to point cloud features at multiple scales. Mask3D sets a new state-of-the-art on ScanNet test (+6.2 mAP), S3DIS 6-fold (+10.1 mAP),LS3D (+11.2 mAP) and ScanNet200 test (+12.4 mAP)
  arXiv  Detail & Related papers  (2022-10-06T17:55:09Z)
• Segmenting Moving Objects via an Object-Centric Layered Representation [100.26138772664811]
  We introduce an object-centric segmentation model with a depth-ordered layer representation. We introduce a scalable pipeline for generating synthetic training data with multiple objects. We evaluate the model on standard video segmentation benchmarks.
  arXiv  Detail & Related papers  (2022-07-05T17:59:43Z)
• INSTA-YOLO: Real-Time Instance Segmentation [2.726684740197893]
  We propose Insta-YOLO, a novel one-stage end-to-end deep learning model for real-time instance segmentation. The proposed model is inspired by the YOLO one-shot object detector, with the box regression loss is replaced with regression in the localization head. We evaluate our model on three datasets, namely, Carnva, Cityscapes and Airbus.
  arXiv  Detail & Related papers  (2021-02-12T21:17:29Z)
• PointINS: Point-based Instance Segmentation [117.38579097923052]
  Mask representation in instance segmentation with Point-of-Interest (PoI) features is challenging because learning a high-dimensional mask feature for each instance requires a heavy computing burden. We propose an instance-aware convolution, which decomposes this mask representation learning task into two tractable modules. Along with instance-aware convolution, we propose PointINS, a simple and practical instance segmentation approach.
  arXiv  Detail & Related papers  (2020-03-13T08:24:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.