Self-supervised Few-shot Learning for Semantic Segmentation: An Annotation-free Approach

• URL: http://arxiv.org/abs/2307.14446v1
• Date: Wed, 26 Jul 2023 18:33:30 GMT
• Title: Self-supervised Few-shot Learning for Semantic Segmentation: An Annotation-free Approach
• Authors: Sanaz Karimijafarbigloo and Reza Azad and Dorit Merhof
• Abstract summary: Few-shot semantic segmentation (FSS) offers immense potential in the field of medical image analysis. Existing FSS techniques heavily rely on annotated semantic classes, rendering them unsuitable for medical images. We propose a novel self-supervised FSS framework that does not rely on any annotation. Instead, it adaptively estimates the query mask by leveraging the eigenvectors obtained from the support images.
• Score: 4.855689194518905
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Few-shot semantic segmentation (FSS) offers immense potential in the field of medical image analysis, enabling accurate object segmentation with limited training data. However, existing FSS techniques heavily rely on annotated semantic classes, rendering them unsuitable for medical images due to the scarcity of annotations. To address this challenge, multiple contributions are proposed: First, inspired by spectral decomposition methods, the problem of image decomposition is reframed as a graph partitioning task. The eigenvectors of the Laplacian matrix, derived from the feature affinity matrix of self-supervised networks, are analyzed to estimate the distribution of the objects of interest from the support images. Secondly, we propose a novel self-supervised FSS framework that does not rely on any annotation. Instead, it adaptively estimates the query mask by leveraging the eigenvectors obtained from the support images. This approach eliminates the need for manual annotation, making it particularly suitable for medical images with limited annotated data. Thirdly, to further enhance the decoding of the query image based on the information provided by the support image, we introduce a multi-scale large kernel attention module. By selectively emphasizing relevant features and details, this module improves the segmentation process and contributes to better object delineation. Evaluations on both natural and medical image datasets demonstrate the efficiency and effectiveness of our method. Moreover, the proposed approach is characterized by its generality and model-agnostic nature, allowing for seamless integration with various deep architectures. The code is publicly available at \href{https://github.com/mindflow-institue/annotation_free_fewshot}{\textcolor{magenta}{GitHub}}.

Related papers

• iSeg: An Iterative Refinement-based Framework for Training-free Segmentation [85.58324416386375]
  We present a deep experimental analysis on iteratively refining cross-attention map with self-attention map. We propose an effective iterative refinement framework for training-free segmentation, named iSeg. Our proposed iSeg achieves an absolute gain of 3.8% in terms of mIoU compared to the best existing training-free approach in literature.
  arXiv  Detail & Related papers  (2024-09-05T03:07:26Z)
• SelfReg-UNet: Self-Regularized UNet for Medical Image Segmentation [2.0792866989795864]
  We explore the patterns learned in a UNet and observe two important factors that potentially affect its performance. We propose to balance the supervision between encoder and decoder and reduce the redundant information in the UNet. The proposed method can be easily integrated into existing UNet architecture in a plug-and-play fashion with negligible computational cost.
  arXiv  Detail & Related papers  (2024-06-21T06:34:56Z)
• Holistic Prototype Attention Network for Few-Shot VOS [74.25124421163542]
  Few-shot video object segmentation (FSVOS) aims to segment dynamic objects of unseen classes by resorting to a small set of support images. We propose a holistic prototype attention network (HPAN) for advancing FSVOS.
  arXiv  Detail & Related papers  (2023-07-16T03:48:57Z)
• Few Shot Medical Image Segmentation with Cross Attention Transformer [30.54965157877615]
  We propose a novel framework for few-shot medical image segmentation, termed CAT-Net. Our proposed network mines the correlations between the support image and query image, limiting them to focus only on useful foreground information. We validated the proposed method on three public datasets: Abd-CT, Abd-MRI, and Card-MRI.
  arXiv  Detail & Related papers  (2023-03-24T09:10:14Z)
• ReFit: A Framework for Refinement of Weakly Supervised Semantic Segmentation using Object Border Fitting for Medical Images [4.945138408504987]
  Weakly Supervised Semantic (WSSS) relying only on image-level supervision is a promising approach to deal with the need for networks. We propose our novel ReFit framework, which deploys state-of-the-art class activation maps combined with various post-processing techniques. By applying our method to WSSS predictions, we achieved up to 10% improvement over the current state-of-the-art WSSS methods for medical imaging.
  arXiv  Detail & Related papers  (2023-03-14T12:46:52Z)
• Self-Supervised Correction Learning for Semi-Supervised Biomedical Image Segmentation [84.58210297703714]
  We propose a self-supervised correction learning paradigm for semi-supervised biomedical image segmentation. We design a dual-task network, including a shared encoder and two independent decoders for segmentation and lesion region inpainting. Experiments on three medical image segmentation datasets for different tasks demonstrate the outstanding performance of our method.
  arXiv  Detail & Related papers  (2023-01-12T08:19:46Z)
• Latent Graph Representations for Critical View of Safety Assessment [2.9724186623561435]
  We propose a method for CVS prediction wherein we first represent a surgical image using a disentangled latent scene graph, then process this representation using a graph neural network. Our graph representations explicitly encode semantic information to improve anatomy-driven reasoning, as well as visual features to retain differentiability and thereby provide robustness to semantic errors. We show that our method not only outperforms several baseline methods when trained with bounding box annotations, but also scales effectively when trained with segmentation masks, maintaining state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-12-08T09:21:09Z)
• Progressively Dual Prior Guided Few-shot Semantic Segmentation [57.37506990980975]
  Few-shot semantic segmentation task aims at performing segmentation in query images with a few annotated support samples. We propose a progressively dual prior guided few-shot semantic segmentation network.
  arXiv  Detail & Related papers  (2022-11-20T16:19:47Z)
• Self-Guided and Cross-Guided Learning for Few-Shot Segmentation [12.899804391102435]
  We propose a self-guided learning approach for few-shot segmentation. By making an initial prediction for the annotated support image, the covered and uncovered foreground regions are encoded to the primary and auxiliary support vectors. By aggregating both primary and auxiliary support vectors, better segmentation performances are obtained on query images.
  arXiv  Detail & Related papers  (2021-03-30T07:36:41Z)
• Self-Supervised Tuning for Few-Shot Segmentation [82.32143982269892]
  Few-shot segmentation aims at assigning a category label to each image pixel with few annotated samples. Existing meta-learning method tends to fail in generating category-specifically discriminative descriptor when the visual features extracted from support images are marginalized in embedding space. This paper presents an adaptive framework tuning, in which the distribution of latent features across different episodes is dynamically adjusted based on a self-segmentation scheme.
  arXiv  Detail & Related papers  (2020-04-12T03:53:53Z)
• High-Order Information Matters: Learning Relation and Topology for Occluded Person Re-Identification [84.43394420267794]
  We propose a novel framework by learning high-order relation and topology information for discriminative features and robust alignment. Our framework significantly outperforms state-of-the-art by6.5%mAP scores on Occluded-Duke dataset.
  arXiv  Detail & Related papers  (2020-03-18T12:18:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.