Cross-Domain Few-Shot Semantic Segmentation via Doubly Matching Transformation

• URL: http://arxiv.org/abs/2405.15265v1
• Date: Fri, 24 May 2024 06:47:43 GMT
• Title: Cross-Domain Few-Shot Semantic Segmentation via Doubly Matching Transformation
• Authors: Jiayi Chen, Rong Quan, Jie Qin,
• Abstract summary: Cross-Domain Few-shot Semantic (CD-FSS) aims to train generalized models that can segment classes from different domains with a few labeled images. Previous works have proven the effectiveness of feature transformation in addressing CD-FSS. We propose a Doubly Matching Transformation-based Network (DMTNet) to solve the above issue.
• Score: 26.788260801305974
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Cross-Domain Few-shot Semantic Segmentation (CD-FSS) aims to train generalized models that can segment classes from different domains with a few labeled images. Previous works have proven the effectiveness of feature transformation in addressing CD-FSS. However, they completely rely on support images for feature transformation, and repeatedly utilizing a few support images for each class may easily lead to overfitting and overlooking intra-class appearance differences. In this paper, we propose a Doubly Matching Transformation-based Network (DMTNet) to solve the above issue. Instead of completely relying on support images, we propose Self-Matching Transformation (SMT) to construct query-specific transformation matrices based on query images themselves to transform domain-specific query features into domain-agnostic ones. Calculating query-specific transformation matrices can prevent overfitting, especially for the meta-testing stage where only one or several images are used as support images to segment hundreds or thousands of images. After obtaining domain-agnostic features, we exploit a Dual Hypercorrelation Construction (DHC) module to explore the hypercorrelations between the query image with the foreground and background of the support image, based on which foreground and background prediction maps are generated and supervised, respectively, to enhance the segmentation result. In addition, we propose a Test-time Self-Finetuning (TSF) strategy to more accurately self-tune the query prediction in unseen domains. Extensive experiments on four popular datasets show that DMTNet achieves superior performance over state-of-the-art approaches. Code is available at https://github.com/ChenJiayi68/DMTNet.

Related papers

• Adapt Before Comparison: A New Perspective on Cross-Domain Few-Shot Segmentation [0.0]
  Cross-domain few-shot segmentation (CD-FSS) has emerged. We show test-time task-adaption is the key for successful CD-FSS. Despite our self-restriction not to use any images other than the few labeled samples at test time, we achieve new state-of-the-art performance in CD-FSS.
  arXiv  Detail & Related papers  (2024-02-27T15:43:53Z)
• Few-shot Medical Image Segmentation via Cross-Reference Transformer [3.2634122554914]
  Few-shot segmentation(FSS) has the potential to address these challenges by learning new categories from a small number of labeled samples. We propose a novel self-supervised few shot medical image segmentation network with Cross-Reference Transformer. Experimental results show that the proposed model achieves good results on both CT dataset and MRI dataset.
  arXiv  Detail & Related papers  (2023-04-19T13:05:18Z)
• I2F: A Unified Image-to-Feature Approach for Domain Adaptive Semantic Segmentation [55.633859439375044]
  Unsupervised domain adaptation (UDA) for semantic segmentation is a promising task freeing people from heavy annotation work. Key idea to tackle this problem is to perform both image-level and feature-level adaptation jointly. This paper proposes a novel UDA pipeline for semantic segmentation that unifies image-level and feature-level adaptation.
  arXiv  Detail & Related papers  (2023-01-03T15:19:48Z)
• Cross-domain Few-shot Segmentation with Transductive Fine-tuning [29.81009103722184]
  We propose to transductively fine-tune the base model on a set of query images under the few-shot setting. Our method could consistently and significantly improve the performance of prototypical FSS models in all cross-domain tasks.
  arXiv  Detail & Related papers  (2022-11-27T06:44:41Z)
• SImProv: Scalable Image Provenance Framework for Robust Content Attribution [80.25476792081403]
  We present SImProv, a framework to match a query image back to a trusted database of originals. SimProv consists of three stages: a scalable search stage for retrieving top-k most similar images; a re-ranking and near-duplicated detection stage for identifying the original among the candidates. We demonstrate effective retrieval and manipulation detection over a dataset of 100 million images.
  arXiv  Detail & Related papers  (2022-06-28T18:42:36Z)
• Boosting Few-shot Semantic Segmentation with Transformers [81.43459055197435]
  TRansformer-based Few-shot Semantic segmentation method (TRFS) Our model consists of two modules: Global Enhancement Module (GEM) and Local Enhancement Module (LEM)
  arXiv  Detail & Related papers  (2021-08-04T20:09:21Z)
• DSP: Dual Soft-Paste for Unsupervised Domain Adaptive Semantic Segmentation [97.74059510314554]
  Unsupervised domain adaptation (UDA) for semantic segmentation aims to adapt a segmentation model trained on the labeled source domain to the unlabeled target domain. Existing methods try to learn domain invariant features while suffering from large domain gaps. We propose a novel Dual Soft-Paste (DSP) method in this paper.
  arXiv  Detail & Related papers  (2021-07-20T16:22:40Z)
• Prototype Mixture Models for Few-shot Semantic Segmentation [50.866870384596446]
  Few-shot segmentation is challenging because objects within the support and query images could significantly differ in appearance and pose. We propose prototype mixture models (PMMs), which correlate diverse image regions with multiple prototypes to enforce the prototype-based semantic representation. PMMs improve 5-shot segmentation performance on MS-COCO by up to 5.82% with only a moderate cost for model size and inference speed.
  arXiv  Detail & Related papers  (2020-08-10T04:33:17Z)
• GMM-UNIT: Unsupervised Multi-Domain and Multi-Modal Image-to-Image Translation via Attribute Gaussian Mixture Modeling [66.50914391679375]
  Unsupervised image-to-image translation (UNIT) aims at learning a mapping between several visual domains by using unpaired training images. Recent studies have shown remarkable success for multiple domains but they suffer from two main limitations. We propose a method named GMM-UNIT, which is based on a content-attribute disentangled representation where the space is fitted with a GMM.
  arXiv  Detail & Related papers  (2020-03-15T10:18:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.