Adapt Before Comparison: A New Perspective on Cross-Domain Few-Shot Segmentation

• URL: http://arxiv.org/abs/2402.17614v2
• Date: Fri, 17 May 2024 17:25:05 GMT
• Title: Adapt Before Comparison: A New Perspective on Cross-Domain Few-Shot Segmentation
• Authors: Jonas Herzog,
• Abstract summary: Cross-domain few-shot segmentation (CD-FSS) has emerged. We show test-time task-adaption is the key for successful CD-FSS. Despite our self-restriction not to use any images other than the few labeled samples at test time, we achieve new state-of-the-art performance in CD-FSS.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Few-shot segmentation performance declines substantially when facing images from a domain different than the training domain, effectively limiting real-world use cases. To alleviate this, recently cross-domain few-shot segmentation (CD-FSS) has emerged. Works that address this task mainly attempted to learn segmentation on a source domain in a manner that generalizes across domains. Surprisingly, we can outperform these approaches while eliminating the training stage and removing their main segmentation network. We show test-time task-adaption is the key for successful CD-FSS instead. Task-adaption is achieved by appending small networks to the feature pyramid of a conventionally classification-pretrained backbone. To avoid overfitting to the few labeled samples in supervised fine-tuning, consistency across augmented views of input images serves as guidance while learning the parameters of the attached layers. Despite our self-restriction not to use any images other than the few labeled samples at test time, we achieve new state-of-the-art performance in CD-FSS, evidencing the need to rethink approaches for the task.

Related papers

• TAVP: Task-Adaptive Visual Prompt for Cross-domain Few-shot Segmentation [44.134340976905655]
  This work proposes a task-adaptive prompt framework based on the Segment Anything Model (SAM) It uses a unique generative approach to prompts alongside a comprehensive model structure and specialized prototype computation. After task-specific and weighted guidance, the abundant feature information of SAM can be better learned for Cross-dominan few-shot segmentation.
  arXiv  Detail & Related papers  (2024-09-09T07:43:58Z)
• Cross-Domain Few-Shot Semantic Segmentation via Doubly Matching Transformation [26.788260801305974]
  Cross-Domain Few-shot Semantic (CD-FSS) aims to train generalized models that can segment classes from different domains with a few labeled images. Previous works have proven the effectiveness of feature transformation in addressing CD-FSS. We propose a Doubly Matching Transformation-based Network (DMTNet) to solve the above issue.
  arXiv  Detail & Related papers  (2024-05-24T06:47:43Z)
• Location-Aware Self-Supervised Transformers [74.76585889813207]
  We propose to pretrain networks for semantic segmentation by predicting the relative location of image parts. We control the difficulty of the task by masking a subset of the reference patch features visible to those of the query. Our experiments show that this location-aware pretraining leads to representations that transfer competitively to several challenging semantic segmentation benchmarks.
  arXiv  Detail & Related papers  (2022-12-05T16:24:29Z)
• Cross-domain Few-shot Segmentation with Transductive Fine-tuning [29.81009103722184]
  We propose to transductively fine-tune the base model on a set of query images under the few-shot setting. Our method could consistently and significantly improve the performance of prototypical FSS models in all cross-domain tasks.
  arXiv  Detail & Related papers  (2022-11-27T06:44:41Z)
• Disentangled Feature Representation for Few-shot Image Classification [64.40410801469106]
  We propose a novel Disentangled Feature Representation framework, dubbed DFR, for few-shot learning applications. DFR can adaptively decouple the discriminative features that are modeled by the classification branch, from the class-irrelevant component of the variation branch. In general, most of the popular deep few-shot learning methods can be plugged in as the classification branch, thus DFR can boost their performance on various few-shot tasks.
  arXiv  Detail & Related papers  (2021-09-26T09:53:11Z)
• Self-Supervision & Meta-Learning for One-Shot Unsupervised Cross-Domain Detection [0.0]
  We present an object detection algorithm able to perform unsupervised adaptation across domains by using only one target sample, seen at test time. We exploit meta-learning to simulate single-sample cross domain learning episodes and better align to the test condition.
  arXiv  Detail & Related papers  (2021-06-07T10:33:04Z)
• Towards Unsupervised Sketch-based Image Retrieval [126.77787336692802]
  We introduce a novel framework that simultaneously performs unsupervised representation learning and sketch-photo domain alignment. Our framework achieves excellent performance in the new unsupervised setting, and performs comparably or better than state-of-the-art in the zero-shot setting.
  arXiv  Detail & Related papers  (2021-05-18T02:38:22Z)
• SCNet: Enhancing Few-Shot Semantic Segmentation by Self-Contrastive Background Prototypes [56.387647750094466]
  Few-shot semantic segmentation aims to segment novel-class objects in a query image with only a few annotated examples. Most of advanced solutions exploit a metric learning framework that performs segmentation through matching each pixel to a learned foreground prototype. This framework suffers from biased classification due to incomplete construction of sample pairs with the foreground prototype only.
  arXiv  Detail & Related papers  (2021-04-19T11:21:47Z)
• A Few Guidelines for Incremental Few-Shot Segmentation [57.34237650765928]
  Given a pretrained segmentation model and few images containing novel classes, our goal is to learn to segment novel classes while retaining the ability to segment previously seen ones. We show how the main problems of end-to-end training in this scenario are. i) the drift of the batch-normalization statistics toward novel classes that we can fix with batch renormalization and. ii) the forgetting of old classes, that we can fix with regularization strategies.
  arXiv  Detail & Related papers  (2020-11-30T20:45:56Z)
• CRNet: Cross-Reference Networks for Few-Shot Segmentation [59.85183776573642]
  Few-shot segmentation aims to learn a segmentation model that can be generalized to novel classes with only a few training images. With a cross-reference mechanism, our network can better find the co-occurrent objects in the two images. Experiments on the PASCAL VOC 2012 dataset show that our network achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-03-24T04:55:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.