Self-supervised Segmentation via Background Inpainting

• URL: http://arxiv.org/abs/2011.05626v1
• Date: Wed, 11 Nov 2020 08:34:40 GMT
• Title: Self-supervised Segmentation via Background Inpainting
• Authors: Isinsu Katircioglu, Helge Rhodin, Victor Constantin, J\"org Sp\"orri, Mathieu Salzmann, Pascal Fua
• Abstract summary: We introduce a self-supervised detection and segmentation approach that can work with single images captured by a potentially moving camera. We exploit a self-supervised loss function that we exploit to train a proposal-based segmentation network. We apply our method to human detection and segmentation in images that visually depart from those of standard benchmarks and outperform existing self-supervised methods.
• Score: 96.10971980098196
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While supervised object detection and segmentation methods achieve impressive accuracy, they generalize poorly to images whose appearance significantly differs from the data they have been trained on. To address this when annotating data is prohibitively expensive, we introduce a self-supervised detection and segmentation approach that can work with single images captured by a potentially moving camera. At the heart of our approach lies the observation that object segmentation and background reconstruction are linked tasks, and that, for structured scenes, background regions can be re-synthesized from their surroundings, whereas regions depicting the moving object cannot. We encode this intuition into a self-supervised loss function that we exploit to train a proposal-based segmentation network. To account for the discrete nature of the proposals, we develop a Monte Carlo-based training strategy that allows the algorithm to explore the large space of object proposals. We apply our method to human detection and segmentation in images that visually depart from those of standard benchmarks and outperform existing self-supervised methods.

Related papers

• Spatial Structure Constraints for Weakly Supervised Semantic Segmentation [100.0316479167605]
  A class activation map (CAM) can only locate the most discriminative part of objects. We propose spatial structure constraints (SSC) for weakly supervised semantic segmentation to alleviate the unwanted object over-activation of attention expansion. Our approach achieves 72.7% and 47.0% mIoU on the PASCAL VOC 2012 and COCO datasets, respectively.
  arXiv  Detail & Related papers  (2024-01-20T05:25:25Z)
• LOCATE: Self-supervised Object Discovery via Flow-guided Graph-cut and Bootstrapped Self-training [13.985488693082981]
  We propose a self-supervised object discovery approach that leverages motion and appearance information to produce high-quality object segmentation masks. We demonstrate the effectiveness of our approach, named LOCATE, on multiple standard video object segmentation, image saliency detection, and object segmentation benchmarks.
  arXiv  Detail & Related papers  (2023-08-22T07:27:09Z)
• Weakly-supervised Contrastive Learning for Unsupervised Object Discovery [52.696041556640516]
  Unsupervised object discovery is promising due to its ability to discover objects in a generic manner. We design a semantic-guided self-supervised learning model to extract high-level semantic features from images. We introduce Principal Component Analysis (PCA) to localize object regions.
  arXiv  Detail & Related papers  (2023-07-07T04:03:48Z)
• Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localization [98.46318529630109]
  We take inspiration from traditional spectral segmentation methods by reframing image decomposition as a graph partitioning problem. We find that these eigenvectors already decompose an image into meaningful segments, and can be readily used to localize objects in a scene. By clustering the features associated with these segments across a dataset, we can obtain well-delineated, nameable regions.
  arXiv  Detail & Related papers  (2022-05-16T17:47:44Z)
• Finding an Unsupervised Image Segmenter in Each of Your Deep Generative Models [92.92095626286223]
  We develop an automatic procedure for finding directions that lead to foreground-background image separation. We use these directions to train an image segmentation model without human supervision.
  arXiv  Detail & Related papers  (2021-05-17T19:34:24Z)
• Self-supervised Human Detection and Segmentation via Multi-view Consensus [116.92405645348185]
  We propose a multi-camera framework in which geometric constraints are embedded in the form of multi-view consistency during training. We show that our approach outperforms state-of-the-art self-supervised person detection and segmentation techniques on images that visually depart from those of standard benchmarks.
  arXiv  Detail & Related papers  (2020-12-09T15:47:21Z)
• Interpretable and Accurate Fine-grained Recognition via Region Grouping [14.28113520947247]
  We present an interpretable deep model for fine-grained visual recognition. At the core of our method lies the integration of region-based part discovery and attribution within a deep neural network. Our results compare favorably to state-of-the-art methods on classification tasks.
  arXiv  Detail & Related papers  (2020-05-21T01:18:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.