UniInst: Unique Representation for End-to-End Instance Segmentation

• URL: http://arxiv.org/abs/2205.12646v2
• Date: Thu, 26 May 2022 04:41:10 GMT
• Title: UniInst: Unique Representation for End-to-End Instance Segmentation
• Authors: Yimin Ou, Rui Yang, Lufan Ma, Yong Liu, Jiangpeng Yan, Shang Xu, Chengjie Wang, Xiu Li
• Abstract summary: We propose a box-free and NMS-free end-to-end instance segmentation framework, termed UniInst. Specifically, we design an instance-aware one-to-one assignment scheme, which dynamically assigns one unique representation to each instance. With these techniques, our UniInst, the first FCN-based end-to-end instance segmentation framework, achieves competitive performance.
• Score: 29.974973664317485
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing instance segmentation methods have achieved impressive performance but still suffer from a common dilemma: redundant representations (e.g., multiple boxes, grids, and anchor points) are inferred for one instance, which leads to multiple duplicated predictions. Thus, mainstream methods usually rely on a hand-designed non-maximum suppression (NMS) post-processing step to select the optimal prediction result, which hinders end-to-end training. To address this issue, we propose a box-free and NMS-free end-to-end instance segmentation framework, termed UniInst, that yields only one unique representation for each instance. Specifically, we design an instance-aware one-to-one assignment scheme, namely Only Yield One Representation (OYOR), which dynamically assigns one unique representation to each instance according to the matching quality between predictions and ground truths. Then, a novel prediction re-ranking strategy is elegantly integrated into the framework to address the misalignment between the classification score and the mask quality, enabling the learned representation to be more discriminative. With these techniques, our UniInst, the first FCN-based end-to-end instance segmentation framework, achieves competitive performance, e.g., 39.0 mask AP using ResNet-50-FPN and 40.2 mask AP using ResNet-101-FPN, against mainstream methods on COCO test-dev. Moreover, the proposed instance-aware method is robust to occlusion scenes, outperforming common baselines by remarkable mask AP on the heavily-occluded OCHuman benchmark. Our codes will be available upon publication.

Related papers

• Bridge the Points: Graph-based Few-shot Segment Anything Semantically [79.1519244940518]
  Recent advancements in pre-training techniques have enhanced the capabilities of vision foundation models. Recent studies extend the SAM to Few-shot Semantic segmentation (FSS) We propose a simple yet effective approach based on graph analysis.
  arXiv  Detail & Related papers  (2024-10-09T15:02:28Z)
• Complete Instances Mining for Weakly Supervised Instance Segmentation [6.177842623752537]
  We propose a novel approach for weakly supervised instance segmentation (WSIS) using only image-level labels. We use MaskIoU heads to predict the integrity scores of proposals and a Complete Instances Mining (CIM) strategy to explicitly model the redundant segmentation problem. Our approach allows the network to become aware of multiple instances and complete instances, and we further improve its robustness through the incorporation of an Anti-noise strategy.
  arXiv  Detail & Related papers  (2024-02-12T13:16:47Z)
• Beyond the Prototype: Divide-and-conquer Proxies for Few-shot Segmentation [63.910211095033596]
  Few-shot segmentation aims to segment unseen-class objects given only a handful of densely labeled samples. We propose a simple yet versatile framework in the spirit of divide-and-conquer. Our proposed approach, named divide-and-conquer proxies (DCP), allows for the development of appropriate and reliable information.
  arXiv  Detail & Related papers  (2022-04-21T06:21:14Z)
• ISTR: End-to-End Instance Segmentation with Transformers [147.14073165997846]
  We propose an instance segmentation Transformer, termed ISTR, which is the first end-to-end framework of its kind. ISTR predicts low-dimensional mask embeddings, and matches them with ground truth mask embeddings for the set loss. Benefiting from the proposed end-to-end mechanism, ISTR demonstrates state-of-the-art performance even with approximation-based suboptimal embeddings.
  arXiv  Detail & Related papers  (2021-05-03T06:00:09Z)
• The Devil is in the Boundary: Exploiting Boundary Representation for Basis-based Instance Segmentation [85.153426159438]
  We propose Basis based Instance(B2Inst) to learn a global boundary representation that can complement existing global-mask-based methods. Our B2Inst leads to consistent improvements and accurately parses out the instance boundaries in a scene.
  arXiv  Detail & Related papers  (2020-11-26T11:26:06Z)
• Mask-guided sample selection for Semi-Supervised Instance Segmentation [13.091166009687058]
  We propose a sample selection approach to decide which samples to annotate for semi-supervised instance segmentation. Our method consists in first predicting pseudo-masks for the unlabeled pool of samples, together with a score predicting the quality of the mask. We study which samples are better to annotate given the quality score, and show how our approach outperforms a random selection.
  arXiv  Detail & Related papers  (2020-08-25T14:44:58Z)
• Mask Encoding for Single Shot Instance Segmentation [97.99956029224622]
  We propose a simple singleshot instance segmentation framework, termed mask encoding based instance segmentation (MEInst) Instead of predicting the two-dimensional mask directly, MEInst distills it into a compact and fixed-dimensional representation vector. We show that the much simpler and flexible one-stage instance segmentation method, can also achieve competitive performance.
  arXiv  Detail & Related papers  (2020-03-26T02:51:17Z)
• PointINS: Point-based Instance Segmentation [117.38579097923052]
  Mask representation in instance segmentation with Point-of-Interest (PoI) features is challenging because learning a high-dimensional mask feature for each instance requires a heavy computing burden. We propose an instance-aware convolution, which decomposes this mask representation learning task into two tractable modules. Along with instance-aware convolution, we propose PointINS, a simple and practical instance segmentation approach.
  arXiv  Detail & Related papers  (2020-03-13T08:24:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.