Sparse Object-level Supervision for Instance Segmentation with Pixel Embeddings

• URL: http://arxiv.org/abs/2103.14572v1
• Date: Fri, 26 Mar 2021 16:36:56 GMT
• Title: Sparse Object-level Supervision for Instance Segmentation with Pixel Embeddings
• Authors: Adrian Wolny, Qin Yu, Constantin Pape, Anna Kreshuk
• Abstract summary: Most state-of-the-art instance segmentation methods have to be trained on densely annotated images. We propose a proposal-free segmentation approach based on non-spatial embeddings. We evaluate the proposed method on challenging 2D and 3D segmentation problems in different microscopy modalities.
• Score: 4.038011160363972
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Most state-of-the-art instance segmentation methods have to be trained on densely annotated images. While difficult in general, this requirement is especially daunting for biomedical images, where domain expertise is often required for annotation. We propose to address the dense annotation bottleneck by introducing a proposal-free segmentation approach based on non-spatial embeddings, which exploits the structure of the learned embedding space to extract individual instances in a differentiable way. The segmentation loss can then be applied directly on the instances and the overall method can be trained on ground truth images where only a few objects are annotated, from scratch or in a semi-supervised transfer learning setting. In addition to the segmentation loss, our setup allows to apply self-supervised consistency losses on the unlabeled parts of the training data. We evaluate the proposed method on challenging 2D and 3D segmentation problems in different microscopy modalities as well as on the popular CVPPP instance segmentation benchmark where we achieve state-of-the-art results. The code is available at: https://github.com/kreshuklab/spoco

Related papers

• Unsupervised Learning of Object-Centric Embeddings for Cell Instance Segmentation in Microscopy Images [3.039768384237206]
  We introduce object-centric embeddings (OCEs) OCEs embed image patches such that the offsets between patches cropped from the same object are preserved. We show theoretically that OCEs can be learnt through a self-supervised task that predicts the spatial offset between image patches.
  arXiv  Detail & Related papers  (2023-10-12T16:59:50Z)
• Weakly Supervised 3D Instance Segmentation without Instance-level Annotations [57.615325809883636]
  3D semantic scene understanding tasks have achieved great success with the emergence of deep learning, but often require a huge amount of manually annotated training data. We propose the first weakly-supervised 3D instance segmentation method that only requires categorical semantic labels as supervision. By generating pseudo instance labels from categorical semantic labels, our designed approach can also assist existing methods for learning 3D instance segmentation at reduced annotation cost.
  arXiv  Detail & Related papers  (2023-08-03T12:30:52Z)
• Synthetic Instance Segmentation from Semantic Image Segmentation Masks [15.477053085267404]
  We propose a novel paradigm called Synthetic Instance (SISeg) SISeg instance segmentation results by leveraging image masks generated by existing semantic segmentation models. In other words, the proposed model does not need extra manpower or higher computational expenses.
  arXiv  Detail & Related papers  (2023-08-02T05:13:02Z)
• Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localization [98.46318529630109]
  We take inspiration from traditional spectral segmentation methods by reframing image decomposition as a graph partitioning problem. We find that these eigenvectors already decompose an image into meaningful segments, and can be readily used to localize objects in a scene. By clustering the features associated with these segments across a dataset, we can obtain well-delineated, nameable regions.
  arXiv  Detail & Related papers  (2022-05-16T17:47:44Z)
• FreeSOLO: Learning to Segment Objects without Annotations [191.82134817449528]
  We present FreeSOLO, a self-supervised instance segmentation framework built on top of the simple instance segmentation method SOLO. Our method also presents a novel localization-aware pre-training framework, where objects can be discovered from complicated scenes in an unsupervised manner.
  arXiv  Detail & Related papers  (2022-02-24T16:31:44Z)
• Learning from Partially Overlapping Labels: Image Segmentation under Annotation Shift [68.6874404805223]
  We propose several strategies for learning from partially overlapping labels in the context of abdominal organ segmentation. We find that combining a semi-supervised approach with an adaptive cross entropy loss can successfully exploit heterogeneously annotated data.
  arXiv  Detail & Related papers  (2021-07-13T09:22:24Z)
• SOLO: A Simple Framework for Instance Segmentation [84.00519148562606]
  "instance categories" assigns categories to each pixel within an instance according to the instance's location. "SOLO" is a simple, direct, and fast framework for instance segmentation with strong performance. Our approach achieves state-of-the-art results for instance segmentation in terms of both speed and accuracy.
  arXiv  Detail & Related papers  (2021-06-30T09:56:54Z)
• Beyond pixel-wise supervision for segmentation: A few global shape descriptors might be surprisingly good! [16.293620755563854]
  Standard losses for training deep segmentation networks could be seen as individual classifications of pixels, instead of supervising the global shape of the predicted segmentations. This study investigates how effective global geometric shape descriptors could be, when used on their own as segmentation losses for training deep networks.
  arXiv  Detail & Related papers  (2021-05-03T13:44:36Z)
• Point-supervised Segmentation of Microscopy Images and Volumes via Objectness Regularization [2.243486411968779]
  This work enables the training of semantic segmentation networks on images with only a single point for training per instance. We achieve competitive results against the state-of-the-art in point-supervised semantic segmentation on challenging datasets in digital pathology.
  arXiv  Detail & Related papers  (2021-03-09T18:40:00Z)
• A Few Guidelines for Incremental Few-Shot Segmentation [57.34237650765928]
  Given a pretrained segmentation model and few images containing novel classes, our goal is to learn to segment novel classes while retaining the ability to segment previously seen ones. We show how the main problems of end-to-end training in this scenario are. i) the drift of the batch-normalization statistics toward novel classes that we can fix with batch renormalization and. ii) the forgetting of old classes, that we can fix with regularization strategies.
  arXiv  Detail & Related papers  (2020-11-30T20:45:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.