Prior to Segment: Foreground Cues for Weakly Annotated Classes in Partially Supervised Instance Segmentation

• URL: http://arxiv.org/abs/2011.11787v2
• Date: Sat, 10 Apr 2021 22:29:26 GMT
• Title: Prior to Segment: Foreground Cues for Weakly Annotated Classes in Partially Supervised Instance Segmentation
• Authors: David Biertimpel, Sindi Shkodrani, Anil S. Baslamisli and N\'ora Baka
• Abstract summary: Partially supervised instance segmentation aims to improve mask prediction with limited mask labels by utilizing the more abundant weak box labels. We show that a class agnostic mask head, commonly used in partially supervised instance segmentation, has difficulties learning a general concept of foreground for the weakly annotated classes. We introduce an object mask prior (OMP) that provides the mask head with the general concept of foreground implicitly learned by the box classification head under the supervision of all classes.
• Score: 3.192503074844774
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Instance segmentation methods require large datasets with expensive and thus limited instance-level mask labels. Partially supervised instance segmentation aims to improve mask prediction with limited mask labels by utilizing the more abundant weak box labels. In this work, we show that a class agnostic mask head, commonly used in partially supervised instance segmentation, has difficulties learning a general concept of foreground for the weakly annotated classes using box supervision only. To resolve this problem we introduce an object mask prior (OMP) that provides the mask head with the general concept of foreground implicitly learned by the box classification head under the supervision of all classes. This helps the class agnostic mask head to focus on the primary object in a region of interest (RoI) and improves generalization to the weakly annotated classes. We test our approach on the COCO dataset using different splits of strongly and weakly supervised classes. Our approach significantly improves over the Mask R-CNN baseline and obtains competitive performance with the state-of-the-art, while offering a much simpler architecture.

Related papers

• MaskUno: Switch-Split Block For Enhancing Instance Segmentation [0.0]
  We propose replacing mask prediction with a Switch-Split block that processes refined ROIs, classifies them, and assigns them to specialized mask predictors. An increase in the mean Average Precision (mAP) of 2.03% was observed for the high-performing DetectoRS when trained on 80 classes.
  arXiv  Detail & Related papers  (2024-07-31T10:12:14Z)
• Mask-free OVIS: Open-Vocabulary Instance Segmentation without Manual Mask Annotations [86.47908754383198]
  Open-Vocabulary (OV) methods leverage large-scale image-caption pairs and vision-language models to learn novel categories. Our method generates pseudo-mask annotations by leveraging the localization ability of a pre-trained vision-language model for objects present in image-caption pairs. Our method trained with just pseudo-masks significantly improves the mAP scores on the MS-COCO dataset and OpenImages dataset.
  arXiv  Detail & Related papers  (2023-03-29T17:58:39Z)
• Discovering Object Masks with Transformers for Unsupervised Semantic Segmentation [75.00151934315967]
  MaskDistill is a novel framework for unsupervised semantic segmentation. Our framework does not latch onto low-level image cues and is not limited to object-centric datasets.
  arXiv  Detail & Related papers  (2022-06-13T17:59:43Z)
• What You See is What You Classify: Black Box Attributions [61.998683569022006]
  We train a deep network, the Explainer, to predict attributions for a pre-trained black-box classifier, the Explanandum. Unlike most existing approaches, ours is capable of directly generating very distinct class-specific masks. We show that our attributions are superior to established methods both visually and quantitatively.
  arXiv  Detail & Related papers  (2022-05-23T12:30:04Z)
• ContrastMask: Contrastive Learning to Segment Every Thing [18.265503138997794]
  We propose ContrastMask, which learns a mask segmentation model on both seen and unseen categories. Features from the mask regions (foreground) are pulled together, and are contrasted against those from the background, and vice versa. Exhaustive experiments on the COCO dataset demonstrate the superiority of our method.
  arXiv  Detail & Related papers  (2022-03-18T07:41:48Z)
• Per-Pixel Classification is Not All You Need for Semantic Segmentation [184.2905747595058]
  Mask classification is sufficiently general to solve both semantic- and instance-level segmentation tasks. We propose MaskFormer, a simple mask classification model which predicts a set of binary masks. Our method outperforms both current state-of-the-art semantic (55.6 mIoU on ADE20K) and panoptic segmentation (52.7 PQ on COCO) models.
  arXiv  Detail & Related papers  (2021-07-13T17:59:50Z)
• SOLO: A Simple Framework for Instance Segmentation [84.00519148562606]
  "instance categories" assigns categories to each pixel within an instance according to the instance's location. "SOLO" is a simple, direct, and fast framework for instance segmentation with strong performance. Our approach achieves state-of-the-art results for instance segmentation in terms of both speed and accuracy.
  arXiv  Detail & Related papers  (2021-06-30T09:56:54Z)
• Weakly-supervised Instance Segmentation via Class-agnostic Learning with Salient Images [34.498542318157284]
  We propose a box-supervised class-agnostic object segmentation (BoxCaseg) based solution for weakly-supervised instance segmentation. The BoxCaseg model is jointly trained using box-supervised images and salient images in a multi-task learning manner. The weakly-supervised Mask R-CNN is on par with fully-supervised Mask R-CNN on PASCAL VOC and significantly outperforms previous state-of-the-art box-supervised instance segmentation methods on COCO.
  arXiv  Detail & Related papers  (2021-04-04T03:01:52Z)
• The surprising impact of mask-head architecture on novel class segmentation [27.076315496682444]
  We show that the architecture of the mask-head plays a surprisingly important role in generalization to classes for which we do not observe masks during training. We also show that the choice of mask-head architecture alone can lead to SOTA results on the partially supervised COCO benchmark without the need of specialty modules or losses proposed by prior literature.
  arXiv  Detail & Related papers  (2021-04-01T16:46:37Z)
• Learning Class-Agnostic Pseudo Mask Generation for Box-Supervised Semantic Segmentation [156.9155100983315]
  We seek for a more accurate learning-based class-agnostic pseudo mask generator tailored to box-supervised semantic segmentation. Our method can further close the performance gap between box-supervised and fully-supervised models.
  arXiv  Detail & Related papers  (2021-03-09T14:54:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.