MaskSplit: Self-supervised Meta-learning for Few-shot Semantic Segmentation

• URL: http://arxiv.org/abs/2110.12207v1
• Date: Sat, 23 Oct 2021 12:30:05 GMT
• Title: MaskSplit: Self-supervised Meta-learning for Few-shot Semantic Segmentation
• Authors: Mustafa Sercan Amac, Ahmet Sencan, Orhun Bugra Baran, Nazli Ikizler-Cinbis, Ramazan Gokberk Cinbis
• Abstract summary: We propose a self-supervised training approach for learning few-shot segmentation models. We first use unsupervised saliency estimation to obtain pseudo-masks on images. We then train a simple prototype based model over different splits of pseudo masks and augmentations of images.
• Score: 10.809349710149533
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Just like other few-shot learning problems, few-shot segmentation aims to minimize the need for manual annotation, which is particularly costly in segmentation tasks. Even though the few-shot setting reduces this cost for novel test classes, there is still a need to annotate the training data. To alleviate this need, we propose a self-supervised training approach for learning few-shot segmentation models. We first use unsupervised saliency estimation to obtain pseudo-masks on images. We then train a simple prototype based model over different splits of pseudo masks and augmentations of images. Our extensive experiments show that the proposed approach achieves promising results, highlighting the potential of self-supervised training. To the best of our knowledge this is the first work that addresses unsupervised few-shot segmentation problem on natural images.

Related papers

• Unsupervised Universal Image Segmentation [59.0383635597103]
  We propose an Unsupervised Universal model (U2Seg) adept at performing various image segmentation tasks. U2Seg generates pseudo semantic labels for these segmentation tasks via leveraging self-supervised models. We then self-train the model on these pseudo semantic labels, yielding substantial performance gains.
  arXiv  Detail & Related papers  (2023-12-28T18:59:04Z)
• Segment, Select, Correct: A Framework for Weakly-Supervised Referring Segmentation [63.13635858586001]
  Referring Image (RIS) is the problem of identifying objects in images through natural language sentences. We propose a novel weakly-supervised framework that tackles RIS by decomposing it into three steps. Using only the first two steps (zero-shot segment and select) outperforms other zero-shot baselines by as much as 16.5%.
  arXiv  Detail & Related papers  (2023-10-20T13:20:17Z)
• Shatter and Gather: Learning Referring Image Segmentation with Text Supervision [52.46081425504072]
  We present a new model that discovers semantic entities in input image and then combines such entities relevant to text query to predict the mask of the referent. Our method was evaluated on four public benchmarks for referring image segmentation, where it clearly outperformed the existing method for the same task and recent open-vocabulary segmentation models on all the benchmarks.
  arXiv  Detail & Related papers  (2023-08-29T15:39:15Z)
• Diffuse, Attend, and Segment: Unsupervised Zero-Shot Segmentation using Stable Diffusion [24.02235805999193]
  We propose a model capable of segmenting anything in a zero-shot manner without any annotations. We introduce a simple yet effective iterative merging process based on measuring KL divergence among attention maps to merge them into valid segmentation masks. On COCO-Stuff-27, our method surpasses the prior unsupervised zero-shot SOTA method by an absolute 26% in pixel accuracy and 17% in mean IoU.
  arXiv  Detail & Related papers  (2023-08-23T23:44:44Z)
• MOCA: Self-supervised Representation Learning by Predicting Masked Online Codebook Assignments [72.6405488990753]
  Self-supervised learning can be used for mitigating the greedy needs of Vision Transformer networks. We propose a single-stage and standalone method, MOCA, which unifies both desired properties. We achieve new state-of-the-art results on low-shot settings and strong experimental results in various evaluation protocols.
  arXiv  Detail & Related papers  (2023-07-18T15:46:20Z)
• One-shot and Partially-Supervised Cell Image Segmentation Using Small Visual Prompt [9.873635079670091]
  We consider an efficient learning framework with as little data as possible. We propose two types of learning strategies: One-shot segmentation which can learn with only one training sample, and Partially-supervised segmentation which assigns annotations to only a part of images. Our proposed methods use a pre-trained model based on only cell images and teach the information of the prompt pairs to the target image.
  arXiv  Detail & Related papers  (2023-04-17T05:04:41Z)
• Contrastive Learning for Self-Supervised Pre-Training of Point Cloud Segmentation Networks With Image Data [7.145862669763328]
  Self-supervised pre-training on unlabelled data is one way to reduce the amount of manual annotations needed. We combine image and point cloud modalities by first learning self-supervised image features and then using these features to train a 3D model. Our pre-training method only requires a single scan of a scene and can be applied to cases where localization information is unavailable.
  arXiv  Detail & Related papers  (2023-01-18T03:14:14Z)
• SCNet: Enhancing Few-Shot Semantic Segmentation by Self-Contrastive Background Prototypes [56.387647750094466]
  Few-shot semantic segmentation aims to segment novel-class objects in a query image with only a few annotated examples. Most of advanced solutions exploit a metric learning framework that performs segmentation through matching each pixel to a learned foreground prototype. This framework suffers from biased classification due to incomplete construction of sample pairs with the foreground prototype only.
  arXiv  Detail & Related papers  (2021-04-19T11:21:47Z)
• Few-Shot Image Classification via Contrastive Self-Supervised Learning [5.878021051195956]
  We propose a new paradigm of unsupervised few-shot learning to repair the deficiencies. We solve the few-shot tasks in two phases: meta-training a transferable feature extractor via contrastive self-supervised learning. Our method achieves state of-the-art performance in a variety of established few-shot tasks on the standard few-shot visual classification datasets.
  arXiv  Detail & Related papers  (2020-08-23T02:24:31Z)
• Single-Stage Semantic Segmentation from Image Labels [25.041129173350104]
  This work defines three desirable properties of a weakly supervised method. We then develop a segmentation-based network model and a self-supervised training scheme to train for semantic masks. Despite its simplicity, our method achieves results that are competitive with significantly more complex pipelines.
  arXiv  Detail & Related papers  (2020-05-16T21:10:10Z)
• CRNet: Cross-Reference Networks for Few-Shot Segmentation [59.85183776573642]
  Few-shot segmentation aims to learn a segmentation model that can be generalized to novel classes with only a few training images. With a cross-reference mechanism, our network can better find the co-occurrent objects in the two images. Experiments on the PASCAL VOC 2012 dataset show that our network achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-03-24T04:55:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.