Task-Adaptive Feature Transformer with Semantic Enrichment for Few-Shot Segmentation

• URL: http://arxiv.org/abs/2202.06498v1
• Date: Mon, 14 Feb 2022 06:16:26 GMT
• Title: Task-Adaptive Feature Transformer with Semantic Enrichment for Few-Shot Segmentation
• Authors: Jun Seo, Young-Hyun Park, Sung Whan Yoon, Jaekyun Moon
• Abstract summary: Few-shot learning allows machines to classify novel classes using only a few labeled samples. We propose a learnable module that can be placed on top of existing segmentation networks for performing few-shot segmentation. Experiments on PASCAL-$5i$ and COCO-$20i$ datasets confirm that the added modules successfully extend the capability of existing segmentators.
• Score: 21.276981570672064
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Few-shot learning allows machines to classify novel classes using only a few labeled samples. Recently, few-shot segmentation aiming at semantic segmentation on low sample data has also seen great interest. In this paper, we propose a learnable module that can be placed on top of existing segmentation networks for performing few-shot segmentation. This module, called the task-adaptive feature transformer (TAFT), linearly transforms task-specific high-level features to a set of task agnostic features well-suited to conducting few-shot segmentation. The task-conditioned feature transformation allows an effective utilization of the semantic information in novel classes to generate tight segmentation masks. We also propose a semantic enrichment (SE) module that utilizes a pixel-wise attention module for high-level feature and an auxiliary loss from an auxiliary segmentation network conducting the semantic segmentation for all training classes. Experiments on PASCAL-$5^i$ and COCO-$20^i$ datasets confirm that the added modules successfully extend the capability of existing segmentators to yield highly competitive few-shot segmentation performances.

Related papers

• CALICO: Part-Focused Semantic Co-Segmentation with Large Vision-Language Models [2.331828779757202]
  We introduce the new task of part-focused semantic co-segmentation, which seeks to identify and segment common and unique objects and parts across images. We present CALICO, the first LVLM that can segment and reason over multiple masks across images, enabling object comparison based on their constituent parts.
  arXiv  Detail & Related papers  (2024-12-26T18:59:37Z)
• Multi-scale Feature Enhancement in Multi-task Learning for Medical Image Analysis [1.6916040234975798]
  Traditional deep learning methods in medical imaging often focus solely on segmentation or classification. We propose a simple yet effective UNet-based MTL model, where features extracted by the encoder are used to predict classification labels, while the decoder produces the segmentation mask. Experimental results across multiple medical datasets confirm the superior performance of our model in both segmentation and classification tasks.
  arXiv  Detail & Related papers  (2024-11-30T04:20:05Z)
• Auxiliary Tasks Enhanced Dual-affinity Learning for Weakly Supervised Semantic Segmentation [79.05949524349005]
  We propose AuxSegNet+, a weakly supervised auxiliary learning framework to explore the rich information from saliency maps. We also propose a cross-task affinity learning mechanism to learn pixel-level affinities from the saliency and segmentation feature maps.
  arXiv  Detail & Related papers  (2024-03-02T10:03:21Z)
• Hierarchical Dense Correlation Distillation for Few-Shot Segmentation-Extended Abstract [47.85056124410376]
  Few-shot semantic segmentation (FSS) aims to form class-agnostic models segmenting unseen classes with only a handful of annotations. We design Hierarchically Decoupled Matching Network (HDMNet) mining pixel-level support correlation based on the transformer architecture. We propose a matching module to reduce train-set overfitting and introduce correlation distillation leveraging semantic correspondence from coarse resolution to boost fine-grained segmentation.
  arXiv  Detail & Related papers  (2023-06-27T08:10:20Z)
• Mask Matching Transformer for Few-Shot Segmentation [71.32725963630837]
  Mask Matching Transformer (MM-Former) is a new paradigm for the few-shot segmentation task. First, the MM-Former follows the paradigm of decompose first and then blend, allowing our method to benefit from the advanced potential objects segmenter. We conduct extensive experiments on the popular COCO-$20i$ and Pascal-$5i$ benchmarks.
  arXiv  Detail & Related papers  (2022-12-05T11:00:32Z)
• Leveraging GAN Priors for Few-Shot Part Segmentation [43.35150430895919]
  Few-shot part segmentation aims to separate different parts of an object given only a few samples. We propose to learn task-specific features in a "pre-training"-"fine-tuning" paradigm.
  arXiv  Detail & Related papers  (2022-07-27T10:17:07Z)
• Boosting Few-shot Semantic Segmentation with Transformers [81.43459055197435]
  TRansformer-based Few-shot Semantic segmentation method (TRFS) Our model consists of two modules: Global Enhancement Module (GEM) and Local Enhancement Module (LEM)
  arXiv  Detail & Related papers  (2021-08-04T20:09:21Z)
• Semantic Attention and Scale Complementary Network for Instance Segmentation in Remote Sensing Images [54.08240004593062]
  We propose an end-to-end multi-category instance segmentation model, which consists of a Semantic Attention (SEA) module and a Scale Complementary Mask Branch (SCMB) SEA module contains a simple fully convolutional semantic segmentation branch with extra supervision to strengthen the activation of interest instances on the feature map. SCMB extends the original single mask branch to trident mask branches and introduces complementary mask supervision at different scales.
  arXiv  Detail & Related papers  (2021-07-25T08:53:59Z)
• Segmenter: Transformer for Semantic Segmentation [79.9887988699159]
  We introduce Segmenter, a transformer model for semantic segmentation. We build on the recent Vision Transformer (ViT) and extend it to semantic segmentation. It outperforms the state of the art on the challenging ADE20K dataset and performs on-par on Pascal Context and Cityscapes.
  arXiv  Detail & Related papers  (2021-05-12T13:01:44Z)
• Task-Adaptive Feature Transformer for Few-Shot Segmentation [21.276981570672064]
  We propose a learnable module for few-shot segmentation, the task-adaptive feature transformer (TAFT) TAFT linearly transforms task-specific high-level features to a set of task-agnostic features well-suited to the segmentation job. Experiments on the PASCAL-$5i$ dataset confirm that this combination successfully adds few-shot learning capability to the segmentation algorithm.
  arXiv  Detail & Related papers  (2020-10-22T04:35:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.