SEMPART: Self-supervised Multi-resolution Partitioning of Image Semantics

• URL: http://arxiv.org/abs/2309.10972v1
• Date: Wed, 20 Sep 2023 00:07:30 GMT
• Title: SEMPART: Self-supervised Multi-resolution Partitioning of Image Semantics
• Authors: Sriram Ravindran, Debraj Basu
• Abstract summary: SEMPART produces high-quality masks rapidly without additional post-processing. Our salient object detection and single object localization findings suggest that SEMPART produces high-quality masks rapidly without additional post-processing.
• Score: 0.5439020425818999
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Accurately determining salient regions of an image is challenging when labeled data is scarce. DINO-based self-supervised approaches have recently leveraged meaningful image semantics captured by patch-wise features for locating foreground objects. Recent methods have also incorporated intuitive priors and demonstrated value in unsupervised methods for object partitioning. In this paper, we propose SEMPART, which jointly infers coarse and fine bi-partitions over an image's DINO-based semantic graph. Furthermore, SEMPART preserves fine boundary details using graph-driven regularization and successfully distills the coarse mask semantics into the fine mask. Our salient object detection and single object localization findings suggest that SEMPART produces high-quality masks rapidly without additional post-processing and benefits from co-optimizing the coarse and fine branches.

Related papers

• Boosting Unsupervised Semantic Segmentation with Principal Mask Proposals [15.258631373740686]
  Unsupervised semantic segmentation aims to automatically partition images into semantically meaningful regions by identifying global semantic categories within an image corpus without any form of annotation. We present PriMaPs - Principal Mask Proposals - decomposing images into semantically meaningful masks based on their feature representation. This allows us to realize unsupervised semantic segmentation by fitting class prototypes to PriMaPs with a expectation-maximization algorithm, PriMaPs-EM.
  arXiv  Detail & Related papers  (2024-04-25T17:58:09Z)
• Variance-insensitive and Target-preserving Mask Refinement for Interactive Image Segmentation [68.16510297109872]
  Point-based interactive image segmentation can ease the burden of mask annotation in applications such as semantic segmentation and image editing. We introduce a novel method, Variance-Insensitive and Target-Preserving Mask Refinement to enhance segmentation quality with fewer user inputs. Experiments on GrabCut, Berkeley, SBD, and DAVIS datasets demonstrate our method's state-of-the-art performance in interactive image segmentation.
  arXiv  Detail & Related papers  (2023-12-22T02:31:31Z)
• MixReorg: Cross-Modal Mixed Patch Reorganization is a Good Mask Learner for Open-World Semantic Segmentation [110.09800389100599]
  We propose MixReorg, a novel and straightforward pre-training paradigm for semantic segmentation. Our approach involves generating fine-grained patch-text pairs data by mixing image patches while preserving the correspondence between patches and text. With MixReorg as a mask learner, conventional text-supervised semantic segmentation models can achieve highly generalizable pixel-semantic alignment ability.
  arXiv  Detail & Related papers  (2023-08-09T09:35:16Z)
• Self-supervised Few-shot Learning for Semantic Segmentation: An Annotation-free Approach [4.855689194518905]
  Few-shot semantic segmentation (FSS) offers immense potential in the field of medical image analysis. Existing FSS techniques heavily rely on annotated semantic classes, rendering them unsuitable for medical images. We propose a novel self-supervised FSS framework that does not rely on any annotation. Instead, it adaptively estimates the query mask by leveraging the eigenvectors obtained from the support images.
  arXiv  Detail & Related papers  (2023-07-26T18:33:30Z)
• ReFit: A Framework for Refinement of Weakly Supervised Semantic Segmentation using Object Border Fitting for Medical Images [4.945138408504987]
  Weakly Supervised Semantic (WSSS) relying only on image-level supervision is a promising approach to deal with the need for networks. We propose our novel ReFit framework, which deploys state-of-the-art class activation maps combined with various post-processing techniques. By applying our method to WSSS predictions, we achieved up to 10% improvement over the current state-of-the-art WSSS methods for medical imaging.
  arXiv  Detail & Related papers  (2023-03-14T12:46:52Z)
• Progressively Dual Prior Guided Few-shot Semantic Segmentation [57.37506990980975]
  Few-shot semantic segmentation task aims at performing segmentation in query images with a few annotated support samples. We propose a progressively dual prior guided few-shot semantic segmentation network.
  arXiv  Detail & Related papers  (2022-11-20T16:19:47Z)
• Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localization [98.46318529630109]
  We take inspiration from traditional spectral segmentation methods by reframing image decomposition as a graph partitioning problem. We find that these eigenvectors already decompose an image into meaningful segments, and can be readily used to localize objects in a scene. By clustering the features associated with these segments across a dataset, we can obtain well-delineated, nameable regions.
  arXiv  Detail & Related papers  (2022-05-16T17:47:44Z)
• Self-Supervised Video Object Segmentation via Cutout Prediction and Tagging [117.73967303377381]
  We propose a novel self-supervised Video Object (VOS) approach that strives to achieve better object-background discriminability. Our approach is based on a discriminative learning loss formulation that takes into account both object and background information. Our proposed approach, CT-VOS, achieves state-of-the-art results on two challenging benchmarks: DAVIS-2017 and Youtube-VOS.
  arXiv  Detail & Related papers  (2022-04-22T17:53:27Z)
• GANSeg: Learning to Segment by Unsupervised Hierarchical Image Generation [16.900404701997502]
  We propose a GAN-based approach that generates images conditioned on latent masks. We show that such mask-conditioned image generation can be learned faithfully when conditioning the masks in a hierarchical manner. It also lets us generate image-mask pairs for training a segmentation network, which outperforms the state-of-the-art unsupervised segmentation methods on established benchmarks.
  arXiv  Detail & Related papers  (2021-12-02T07:57:56Z)
• Causal Intervention for Weakly-Supervised Semantic Segmentation [122.1846968696862]
  We aim to generate better pixel-level pseudo-masks by using only image-level labels. We propose a structural causal model to analyze the causalities among images, contexts, and class labels. Based on it, we develop a new method: Context Adjustment (CONTA), to remove the confounding bias in image-level classification.
  arXiv  Detail & Related papers  (2020-09-26T09:26:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.