Few-shot Segmentation with Optimal Transport Matching and Message Flow

• URL: http://arxiv.org/abs/2108.08518v1
• Date: Thu, 19 Aug 2021 06:26:11 GMT
• Title: Few-shot Segmentation with Optimal Transport Matching and Message Flow
• Authors: Weide Liu, Chi Zhang, Henghui Ding, Tzu-Yi Hung and Guosheng Lin
• Abstract summary: It is essential for few-shot semantic segmentation to fully utilize the support information. We propose a Correspondence Matching Network (CMNet) with an Optimal Transport Matching module. Experiments on PASCAL VOC 2012, MS COCO, and FSS-1000 datasets show that our network achieves new state-of-the-art few-shot segmentation performance.
• Score: 50.9853556696858
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We address the challenging task of few-shot segmentation in this work. It is essential for few-shot semantic segmentation to fully utilize the support information. Previous methods typically adapt masked average pooling over the support feature to extract the support clues as a global vector, usually dominated by the salient part and loses some important clues. In this work, we argue that every support pixel's information is desired to be transferred to all query pixels and propose a Correspondence Matching Network (CMNet) with an Optimal Transport Matching module to mine out the correspondence between the query and support images. Besides, it is important to fully utilize both local and global information from the annotated support images. To this end, we propose a Message Flow module to propagate the message along the inner-flow within the same image and cross-flow between support and query images, which greatly help enhance the local feature representations. We further address the few-shot segmentation as a multi-task learning problem to alleviate the domain gap issue between different datasets. Experiments on PASCAL VOC 2012, MS COCO, and FSS-1000 datasets show that our network achieves new state-of-the-art few-shot segmentation performance.

Related papers

• MatchSeg: Towards Better Segmentation via Reference Image Matching [5.55078598520531]
  Few-shot learning aims to overcome the need for annotated data by using a small labeled dataset, known as a support set, to guide predicting labels for new, unlabeled images. Inspired by this paradigm, we introduce MatchSeg, a novel framework that enhances medical image segmentation through strategic reference image matching.
  arXiv  Detail & Related papers  (2024-03-23T18:04:58Z)
• Multi-interactive Feature Learning and a Full-time Multi-modality Benchmark for Image Fusion and Segmentation [66.15246197473897]
  Multi-modality image fusion and segmentation play a vital role in autonomous driving and robotic operation. We propose a textbfMulti-textbfinteractive textbfFeature learning architecture for image fusion and textbfSegmentation.
  arXiv  Detail & Related papers  (2023-08-04T01:03:58Z)
• Image-Specific Information Suppression and Implicit Local Alignment for Text-based Person Search [61.24539128142504]
  Text-based person search (TBPS) is a challenging task that aims to search pedestrian images with the same identity from an image gallery given a query text. Most existing methods rely on explicitly generated local parts to model fine-grained correspondence between modalities. We propose an efficient joint Multi-level Alignment Network (MANet) for TBPS, which can learn aligned image/text feature representations between modalities at multiple levels.
  arXiv  Detail & Related papers  (2022-08-30T16:14:18Z)
• Dense Cross-Query-and-Support Attention Weighted Mask Aggregation for Few-Shot Segmentation [25.605580031284052]
  Few-shot Semantic Dense (FSS) has attracted great attention. The goal of FSS is to segment target objects in a query image given only a few annotated support images of the target class. We propose pixel-wise Cross-query-and-support Attention weighted Mask Aggregation (AMADC), where both foreground and background support information are fully exploited.
  arXiv  Detail & Related papers  (2022-07-18T12:12:42Z)
• Few-Shot Segmentation with Global and Local Contrastive Learning [51.677179037590356]
  We propose a prior extractor to learn the query information from the unlabeled images with our proposed global-local contrastive learning. We generate the prior region maps for query images, which locate the objects, as guidance to perform cross interaction with support features. Without bells and whistles, the proposed approach achieves new state-of-the-art performance for the few-shot segmentation task.
  arXiv  Detail & Related papers  (2021-08-11T15:52:22Z)
• Boosting Few-shot Semantic Segmentation with Transformers [81.43459055197435]
  TRansformer-based Few-shot Semantic segmentation method (TRFS) Our model consists of two modules: Global Enhancement Module (GEM) and Local Enhancement Module (LEM)
  arXiv  Detail & Related papers  (2021-08-04T20:09:21Z)
• Self-Guided and Cross-Guided Learning for Few-Shot Segmentation [12.899804391102435]
  We propose a self-guided learning approach for few-shot segmentation. By making an initial prediction for the annotated support image, the covered and uncovered foreground regions are encoded to the primary and auxiliary support vectors. By aggregating both primary and auxiliary support vectors, better segmentation performances are obtained on query images.
  arXiv  Detail & Related papers  (2021-03-30T07:36:41Z)
• CRNet: Cross-Reference Networks for Few-Shot Segmentation [59.85183776573642]
  Few-shot segmentation aims to learn a segmentation model that can be generalized to novel classes with only a few training images. With a cross-reference mechanism, our network can better find the co-occurrent objects in the two images. Experiments on the PASCAL VOC 2012 dataset show that our network achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-03-24T04:55:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.