ISDA: Position-Aware Instance Segmentation with Deformable Attention

• URL: http://arxiv.org/abs/2202.12251v1
• Date: Wed, 23 Feb 2022 12:30:18 GMT
• Title: ISDA: Position-Aware Instance Segmentation with Deformable Attention
• Authors: Kaining Ying, Zhenhua Wang, Cong Bai, Pengfei Zhou
• Abstract summary: We propose a novel end-to-end instance segmentation method termed ISDA. It reshapes the task into predicting a set of object masks, which are generated via traditional convolution operation. Thanks to the introduced set-prediction mechanism, the proposed method is NMS-free.
• Score: 4.188555841288538
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Most instance segmentation models are not end-to-end trainable due to either the incorporation of proposal estimation (RPN) as a pre-processing or non-maximum suppression (NMS) as a post-processing. Here we propose a novel end-to-end instance segmentation method termed ISDA. It reshapes the task into predicting a set of object masks, which are generated via traditional convolution operation with learned position-aware kernels and features of objects. Such kernels and features are learned by leveraging a deformable attention network with multi-scale representation. Thanks to the introduced set-prediction mechanism, the proposed method is NMS-free. Empirically, ISDA outperforms Mask R-CNN (the strong baseline) by 2.6 points on MS-COCO, and achieves leading performance compared with recent models. Code will be available soon.

Related papers

• MaskUno: Switch-Split Block For Enhancing Instance Segmentation [0.0]
  We propose replacing mask prediction with a Switch-Split block that processes refined ROIs, classifies them, and assigns them to specialized mask predictors. An increase in the mean Average Precision (mAP) of 2.03% was observed for the high-performing DetectoRS when trained on 80 classes.
  arXiv  Detail & Related papers  (2024-07-31T10:12:14Z)
• Complete Instances Mining for Weakly Supervised Instance Segmentation [6.177842623752537]
  We propose a novel approach for weakly supervised instance segmentation (WSIS) using only image-level labels. We use MaskIoU heads to predict the integrity scores of proposals and a Complete Instances Mining (CIM) strategy to explicitly model the redundant segmentation problem. Our approach allows the network to become aware of multiple instances and complete instances, and we further improve its robustness through the incorporation of an Anti-noise strategy.
  arXiv  Detail & Related papers  (2024-02-12T13:16:47Z)
• UniInst: Unique Representation for End-to-End Instance Segmentation [29.974973664317485]
  We propose a box-free and NMS-free end-to-end instance segmentation framework, termed UniInst. Specifically, we design an instance-aware one-to-one assignment scheme, which dynamically assigns one unique representation to each instance. With these techniques, our UniInst, the first FCN-based end-to-end instance segmentation framework, achieves competitive performance.
  arXiv  Detail & Related papers  (2022-05-25T10:40:26Z)
• Sparse Instance Activation for Real-Time Instance Segmentation [72.23597664935684]
  We propose a conceptually novel, efficient, and fully convolutional framework for real-time instance segmentation. SparseInst has extremely fast inference speed and achieves 40 FPS and 37.9 AP on the COCO benchmark.
  arXiv  Detail & Related papers  (2022-03-24T03:15:39Z)
• End-to-End Object Detection with Fully Convolutional Network [71.56728221604158]
  We introduce a Prediction-aware One-To-One (POTO) label assignment for classification to enable end-to-end detection. A simple 3D Max Filtering (3DMF) is proposed to utilize the multi-scale features and improve the discriminability of convolutions in the local region. Our end-to-end framework achieves competitive performance against many state-of-the-art detectors with NMS on COCO and CrowdHuman datasets.
  arXiv  Detail & Related papers  (2020-12-07T09:14:55Z)
• Deep Variational Instance Segmentation [7.334808870313923]
  State-of-the-art algorithms often employ two separate stages, the first one generating object proposals and the second one recognizing and refining the boundaries. We propose a novel algorithm that directly utilizes a fully convolutional network (FCN) to predict instance labels.
  arXiv  Detail & Related papers  (2020-07-22T17:57:49Z)
• Boundary-assisted Region Proposal Networks for Nucleus Segmentation [89.69059532088129]
  Machine learning models cannot perform well because of large amount of crowded nuclei. We devise a Boundary-assisted Region Proposal Network (BRP-Net) that achieves robust instance-level nucleus segmentation.
  arXiv  Detail & Related papers  (2020-06-04T08:26:38Z)
• SOLOv2: Dynamic and Fast Instance Segmentation [102.15325936477362]
  We build a simple, direct, and fast instance segmentation framework with strong performance. We take one step further by dynamically learning the mask head of the object segmenter. We demonstrate a simple direct instance segmentation system, outperforming a few state-of-the-art methods in both speed and accuracy.
  arXiv  Detail & Related papers  (2020-03-23T09:44:21Z)
• PointINS: Point-based Instance Segmentation [117.38579097923052]
  Mask representation in instance segmentation with Point-of-Interest (PoI) features is challenging because learning a high-dimensional mask feature for each instance requires a heavy computing burden. We propose an instance-aware convolution, which decomposes this mask representation learning task into two tractable modules. Along with instance-aware convolution, we propose PointINS, a simple and practical instance segmentation approach.
  arXiv  Detail & Related papers  (2020-03-13T08:24:58Z)
• Conditional Convolutions for Instance Segmentation [109.2706837177222]
  We propose a simple yet effective instance segmentation framework, termed CondInst. We employ dynamic instance-aware networks, conditioned on instances. We demonstrate a simpler instance segmentation method that can achieve improved performance in both accuracy and inference speed.
  arXiv  Detail & Related papers  (2020-03-12T08:42:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.