Judging from Support-set: A New Way to Utilize Few-Shot Segmentation for Segmentation Refinement Process

• URL: http://arxiv.org/abs/2407.04519v2
• Date: Thu, 10 Oct 2024 04:24:21 GMT
• Title: Judging from Support-set: A New Way to Utilize Few-Shot Segmentation for Segmentation Refinement Process
• Authors: Seonghyeon Moon, Qingze, Liu, Haein Kong, Muhammad Haris Khan,
• Abstract summary: segmentation refinement aims to enhance the initial coarse masks generated by segmentation algorithms. No method has been developed that can determine the success of segmentation refinement. We propose Judging From Support-set (JFS), a method to judge the success of segmentation refinement.
• Score: 8.407954312239454
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Segmentation refinement aims to enhance the initial coarse masks generated by segmentation algorithms. The refined masks are expected to capture more details and better contours of the target objects. Research on segmentation refinement has developed as a response to the need for high-quality image segmentations. However, to our knowledge, no method has been developed that can determine the success of segmentation refinement. Such a method could ensure the reliability of segmentation in applications where the outcome of the segmentation is important and fosters innovation in image processing technologies. To address this research gap, we propose Judging From Support-set (JFS), a method to judge the success of segmentation refinement leveraging an off-the-shelf few-shot segmentation (FSS) model. The traditional goal of the problem in FSS is to find a target object in a query image utilizing target information given by a support set. However, we propose a novel application of the FSS model in our evaluation pipeline for segmentation refinement methods. Given a coarse mask as input, segmentation refinement methods produce a refined mask; these two masks become new support masks for the FSS model. The existing support mask then serves as the test set for the FSS model to evaluate the quality of the refined segmentation by the segmentation refinement methods.We demonstrate the effectiveness of our proposed JFS framework by evaluating the SAM Enhanced Pseduo-Labels (SEPL) using SegGPT as the choice of FSS model on the PASCAL dataset. The results showed that JFS has the potential to determine whether the segmentation refinement process is successful.

Related papers

• Freestyle Sketch-in-the-Loop Image Segmentation [116.1810651297801]
  We introduce a "sketch-in-the-loop" image segmentation framework, enabling the segmentation of visual concepts partially, completely, or in groupings. This framework capitalises on the synergy between sketch-based image retrieval models and large-scale pre-trained models. Our purpose-made augmentation strategy enhances the versatility of our sketch-guided mask generation, allowing segmentation at multiple levels.
  arXiv  Detail & Related papers  (2025-01-27T13:07:51Z)
• Bridge the Points: Graph-based Few-shot Segment Anything Semantically [79.1519244940518]
  Recent advancements in pre-training techniques have enhanced the capabilities of vision foundation models. Recent studies extend the SAM to Few-shot Semantic segmentation (FSS) We propose a simple yet effective approach based on graph analysis.
  arXiv  Detail & Related papers  (2024-10-09T15:02:28Z)
• PosSAM: Panoptic Open-vocabulary Segment Anything [58.72494640363136]
  PosSAM is an open-vocabulary panoptic segmentation model that unifies the strengths of the Segment Anything Model (SAM) with the vision-native CLIP model in an end-to-end framework. We introduce a Mask-Aware Selective Ensembling (MASE) algorithm that adaptively enhances the quality of generated masks and boosts the performance of open-vocabulary classification during inference for each image.
  arXiv  Detail & Related papers  (2024-03-14T17:55:03Z)
• Boosting Few-Shot Segmentation via Instance-Aware Data Augmentation and Local Consensus Guided Cross Attention [7.939095881813804]
  Few-shot segmentation aims to train a segmentation model that can fast adapt to a novel task for which only a few annotated images are provided. We introduce an instance-aware data augmentation (IDA) strategy that augments the support images based on the relative sizes of the target objects. The proposed IDA effectively increases the support set's diversity and promotes the distribution consistency between support and query images.
  arXiv  Detail & Related papers  (2024-01-18T10:29:10Z)
• SegRefiner: Towards Model-Agnostic Segmentation Refinement with Discrete Diffusion Process [102.18226145874007]
  We propose a model-agnostic solution called SegRefiner to enhance the quality of object masks produced by different segmentation models. SegRefiner takes coarse masks as inputs and refines them using a discrete diffusion process. It consistently improves both the segmentation metrics and boundary metrics across different types of coarse masks.
  arXiv  Detail & Related papers  (2023-12-19T18:53:47Z)
• Mask Matching Transformer for Few-Shot Segmentation [71.32725963630837]
  Mask Matching Transformer (MM-Former) is a new paradigm for the few-shot segmentation task. First, the MM-Former follows the paradigm of decompose first and then blend, allowing our method to benefit from the advanced potential objects segmenter. We conduct extensive experiments on the popular COCO-$20i$ and Pascal-$5i$ benchmarks.
  arXiv  Detail & Related papers  (2022-12-05T11:00:32Z)
• HMFS: Hybrid Masking for Few-Shot Segmentation [27.49000348046462]
  We develop a simple, effective, and efficient approach to enhance feature masking (FM) We compensate for the loss of fine-grained spatial details in FM technique by investigating and leveraging a complementary basic input masking method. Experimental results on three publicly available benchmarks reveal that HMFS outperforms the current state-of-the-art methods by visible margins.
  arXiv  Detail & Related papers  (2022-03-24T03:07:20Z)
• RefineMask: Towards High-Quality Instance Segmentation with Fine-Grained Features [53.71163467683838]
  RefineMask is a new method for high-quality instance segmentation of objects and scenes. It incorporates fine-grained features during the instance-wise segmenting process in a multi-stage manner. It succeeds in segmenting hard cases such as bent parts of objects that are over-smoothed by most previous methods.
  arXiv  Detail & Related papers  (2021-04-17T15:09:20Z)
• SimPropNet: Improved Similarity Propagation for Few-shot Image Segmentation [14.419517737536706]
  Recent deep neural network based FSS methods leverage high-dimensional feature similarity between the foreground features of the support images and the query image features. We propose to jointly predict the support and query masks to force the support features to share characteristics with the query features. Our method achieves state-of-the-art results for one-shot and five-shot segmentation on the PASCAL-5i dataset.
  arXiv  Detail & Related papers  (2020-04-30T17:56:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.