CompetitorFormer: Competitor Transformer for 3D Instance Segmentation

• URL: http://arxiv.org/abs/2411.14179v3
• Date: Fri, 03 Jan 2025 04:54:56 GMT
• Title: CompetitorFormer: Competitor Transformer for 3D Instance Segmentation
• Abstract summary: Transformer-based methods have become the dominant approach for 3D instance segmentation.<n>Current models employ a fixed and higher number of queries than the instances present within a scene.<n>In such instances, multiple queries predict the same instance, yet only a single query is ultimately optimized.
• Score: 8.201162826445252
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Transformer-based methods have become the dominant approach for 3D instance segmentation. These methods predict instance masks via instance queries, ranking them by classification confidence and IoU scores to select the top prediction as the final outcome. However, it has been observed that the current models employ a fixed and higher number of queries than the instances present within a scene. In such instances, multiple queries predict the same instance, yet only a single query is ultimately optimized. The close scores of queries in the lower-level decoders make it challenging for the dominant query to distinguish itself rapidly, which ultimately impairs the model's accuracy and convergence efficiency. This phenomenon is referred to as inter-query competition. To address this challenge, we put forth a series of plug-and-play competition-oriented designs, collectively designated as the CompetitorFormer, with the aim of reducing competition and facilitating a dominant query. Experiments showed that integrating our designs with state-of-the-art frameworks consistently resulted in significant performance improvements in 3D instance segmentation across a range of datasets.

Related papers

• GVC-Seg: Training-Free 3D Instance Segmentation via Geometric Visual Correspondence [50.10864740041483]
  We propose a training-free 3D instance segmentation approach via Geometric Visual Correspondence (GVC-Seg)<n>GVC-Seg exploits the correspondence between 3D geometric cues and 2D visual cues to mitigate the confidence bias.<n>Our method achieves state-of-the-art performance on several challenging benchmarks, while also exhibiting strong potential in open-vocabulary semantic segmentation settings.
  arXiv  Detail & Related papers  (2026-06-06T07:09:15Z)
• MIQ-SAM3D: From Single-Point Prompt to Multi-Instance Segmentation via Competitive Query Refinement [0.328418927821443]
  We propose MIQ-SAM3D, a multi-instance 3D segmentation framework with a competitive query optimization strategy.<n>A prompt-conditioned instance-query generator transforms a single point prompt into multiple specialized queries.<n>A hybrid CNN-Transformer encoder injects CNN-derived boundary saliency into ViT self-attention via spatial gating.<n>A competitively optimized query decoder then enables end-to-end, parallel, multi-instance prediction.
  arXiv  Detail & Related papers  (2025-11-03T08:48:28Z)
• Test-Time Consistency in Vision Language Models [26.475993408532304]
  Vision-Language Models (VLMs) have achieved impressive performance across a wide range of multimodal tasks.<n>Recent benchmarks, such as MM-R3, highlight that even state-of-the-art VLMs can produce divergent predictions across semantically equivalent inputs.<n>We propose a simple and effective test-time consistency framework that enhances semantic consistency without supervised re-training.
  arXiv  Detail & Related papers  (2025-06-27T17:09:44Z)
• CLIMB-3D: Continual Learning for Imbalanced 3D Instance Segmentation [67.36817440834251]
  We propose a unified framework for textbfCLass-incremental textbfImbalance-aware textbf3DIS.<n>Our approach achieves state-of-the-art results, surpassing prior work by up to 16.76% mAP for instance segmentation and approximately 30% mIoU for semantic segmentation.
  arXiv  Detail & Related papers  (2025-02-24T18:58:58Z)
• SGIFormer: Semantic-guided and Geometric-enhanced Interleaving Transformer for 3D Instance Segmentation [14.214197948110115]
  This paper introduces a novel method, named SGIFormer, for 3D instance segmentation. It is composed of the Semantic-guided Mix Query (SMQ) and the Geometric-enhanced Interleaving Transformer (GIT) decoder. It attains state-of-the-art performance on ScanNet V2, ScanNet200, and the challenging high-fidelity ScanNet++ benchmark.
  arXiv  Detail & Related papers  (2024-07-16T10:17:28Z)
• EipFormer: Emphasizing Instance Positions in 3D Instance Segmentation [51.996943482875366]
  We present a novel Transformer-based architecture, EipFormer, which comprises progressive aggregation and dual position embedding. EipFormer achieves superior or comparable performance compared to state-of-the-art approaches.
  arXiv  Detail & Related papers  (2023-12-09T16:08:47Z)
• Tackling Diverse Minorities in Imbalanced Classification [80.78227787608714]
  Imbalanced datasets are commonly observed in various real-world applications, presenting significant challenges in training classifiers. We propose generating synthetic samples iteratively by mixing data samples from both minority and majority classes. We demonstrate the effectiveness of our proposed framework through extensive experiments conducted on seven publicly available benchmark datasets.
  arXiv  Detail & Related papers  (2023-08-28T18:48:34Z)
• Domain Generalization via Rationale Invariance [70.32415695574555]
  This paper offers a new perspective to ease the challenge of domain generalization, which involves maintaining robust results even in unseen environments. We propose treating the element-wise contributions to the final results as the rationale for making a decision and representing the rationale for each sample as a matrix. Our experiments demonstrate that the proposed approach achieves competitive results across various datasets, despite its simplicity.
  arXiv  Detail & Related papers  (2023-08-22T03:31:40Z)
• Spatial-Temporal Graph Enhanced DETR Towards Multi-Frame 3D Object Detection [54.041049052843604]
  We present STEMD, a novel end-to-end framework that enhances the DETR-like paradigm for multi-frame 3D object detection. First, to model the inter-object spatial interaction and complex temporal dependencies, we introduce the spatial-temporal graph attention network. Finally, it poses a challenge for the network to distinguish between the positive query and other highly similar queries that are not the best match.
  arXiv  Detail & Related papers  (2023-07-01T13:53:14Z)
• 3D-QueryIS: A Query-based Framework for 3D Instance Segmentation [74.6998931386331]
  Previous methods for 3D instance segmentation often maintain inter-task dependencies and the tendency towards a lack of robustness. We propose a novel query-based method, termed as 3D-QueryIS, which is detector-free, semantic segmentation-free, and cluster-free. Our 3D-QueryIS is free from the accumulated errors caused by the inter-task dependencies.
  arXiv  Detail & Related papers  (2022-11-17T07:04:53Z)
• UniInst: Unique Representation for End-to-End Instance Segmentation [29.974973664317485]
  We propose a box-free and NMS-free end-to-end instance segmentation framework, termed UniInst. Specifically, we design an instance-aware one-to-one assignment scheme, which dynamically assigns one unique representation to each instance. With these techniques, our UniInst, the first FCN-based end-to-end instance segmentation framework, achieves competitive performance.
  arXiv  Detail & Related papers  (2022-05-25T10:40:26Z)
• PointInst3D: Segmenting 3D Instances by Points [136.7261709896713]
  We propose a fully-convolutional 3D point cloud instance segmentation method that works in a per-point prediction fashion. We find the key to its success is assigning a suitable target to each sampled point. Our approach achieves promising results on both ScanNet and S3DIS benchmarks.
  arXiv  Detail & Related papers  (2022-04-25T02:41:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.