Modular Adaptation for Cross-Domain Few-Shot Learning

• URL: http://arxiv.org/abs/2104.00619v1
• Date: Thu, 1 Apr 2021 16:50:43 GMT
• Title: Modular Adaptation for Cross-Domain Few-Shot Learning
• Authors: Xiao Lin, Meng Ye, Yunye Gong, Giedrius Buracas, Nikoletta Basiou, Ajay Divakaran, Yi Yao
• Abstract summary: We show that substantial performance improvement of downstream tasks can be achieved by appropriate designs of the adaptation process. We propose a modular adaptation method that selectively performs multiple state-of-the-art (SOTA) adaptation methods in sequence. As different downstream tasks may require different types of adaptation, our modular adaptation enables the dynamic configuration of the most suitable modules.
• Score: 8.997255739981437
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Adapting pre-trained representations has become the go-to recipe for learning new downstream tasks with limited examples. While literature has demonstrated great successes via representation learning, in this work, we show that substantial performance improvement of downstream tasks can also be achieved by appropriate designs of the adaptation process. Specifically, we propose a modular adaptation method that selectively performs multiple state-of-the-art (SOTA) adaptation methods in sequence. As different downstream tasks may require different types of adaptation, our modular adaptation enables the dynamic configuration of the most suitable modules based on the downstream task. Moreover, as an extension to existing cross-domain 5-way k-shot benchmarks (e.g., miniImageNet -> CUB), we create a new high-way (~100) k-shot benchmark with data from 10 different datasets. This benchmark provides a diverse set of domains and allows the use of stronger representations learned from ImageNet. Experimental results show that by customizing adaptation process towards downstream tasks, our modular adaptation pipeline (MAP) improves 3.1% in 5-shot classification accuracy over baselines of finetuning and Prototypical Networks.

Related papers

• Context-Aware Multimodal Pretraining [72.04020920042574]
  We show that vision-language models can be trained to exhibit significantly increased few-shot adaptation. We find up to four-fold improvements in test-time sample efficiency, and average few-shot adaptation gains of over 5%.
  arXiv  Detail & Related papers  (2024-11-22T17:55:39Z)
• Adaptive FSS: A Novel Few-Shot Segmentation Framework via Prototype Enhancement [6.197356908000006]
  The Few-Shot (FSS) aims to accomplish the novel class segmentation task with a few annotated images. We propose a novel framework based on the adapter mechanism, namely Adaptive FSS, which can efficiently adapt the existing FSS model to the novel classes. Our approach is compatible with diverse FSS methods with different backbones by simply inserting PAM between the layers of the encoder.
  arXiv  Detail & Related papers  (2023-12-25T14:03:38Z)
• Informative Data Mining for One-Shot Cross-Domain Semantic Segmentation [84.82153655786183]
  We propose a novel framework called Informative Data Mining (IDM) to enable efficient one-shot domain adaptation for semantic segmentation. IDM provides an uncertainty-based selection criterion to identify the most informative samples, which facilitates quick adaptation and reduces redundant training. Our approach outperforms existing methods and achieves a new state-of-the-art one-shot performance of 56.7%/55.4% on the GTA5/SYNTHIA to Cityscapes adaptation tasks.
  arXiv  Detail & Related papers  (2023-09-25T15:56:01Z)
• Multi-Domain Learning with Modulation Adapters [33.54630534228469]
  Multi-domain learning aims to handle related tasks, such as image classification across multiple domains, simultaneously. Modulation Adapters update the convolutional weights of the model in a multiplicative manner for each task. Our approach yields excellent results, with accuracies that are comparable to or better than those of existing state-of-the-art approaches.
  arXiv  Detail & Related papers  (2023-07-17T14:40:16Z)
• Benchmarking Test-Time Adaptation against Distribution Shifts in Image Classification [77.0114672086012]
  Test-time adaptation (TTA) is a technique aimed at enhancing the generalization performance of models by leveraging unlabeled samples solely during prediction. We present a benchmark that systematically evaluates 13 prominent TTA methods and their variants on five widely used image classification datasets.
  arXiv  Detail & Related papers  (2023-07-06T16:59:53Z)
• Adaptive Spot-Guided Transformer for Consistent Local Feature Matching [64.30749838423922]
  We propose Adaptive Spot-Guided Transformer (ASTR) for local feature matching. ASTR models the local consistency and scale variations in a unified coarse-to-fine architecture.
  arXiv  Detail & Related papers  (2023-03-29T12:28:01Z)
• Few-Shot Adaptation of Pre-Trained Networks for Domain Shift [17.123505029637055]
  Deep networks are prone to performance degradation when there is a domain shift between the source (training) data and target (test) data. Recent test-time adaptation methods update batch normalization layers of pre-trained source models deployed in new target environments with streaming data to mitigate such performance degradation. We propose a framework for few-shot domain adaptation to address the practical challenges of data-efficient adaptation.
  arXiv  Detail & Related papers  (2022-05-30T16:49:59Z)
• Task Adaptive Parameter Sharing for Multi-Task Learning [114.80350786535952]
  Adaptive Task Adapting Sharing (TAPS) is a method for tuning a base model to a new task by adaptively modifying a small, task-specific subset of layers. Compared to other methods, TAPS retains high accuracy on downstream tasks while introducing few task-specific parameters. We evaluate our method on a suite of fine-tuning tasks and architectures (ResNet, DenseNet, ViT) and show that it achieves state-of-the-art performance while being simple to implement.
  arXiv  Detail & Related papers  (2022-03-30T23:16:07Z)
• Learning Instance-Specific Adaptation for Cross-Domain Segmentation [79.61787982393238]
  We propose a test-time adaptation method for cross-domain image segmentation. Given a new unseen instance at test time, we adapt a pre-trained model by conducting instance-specific BatchNorm calibration.
  arXiv  Detail & Related papers  (2022-03-30T17:59:45Z)
• Parameter-efficient Model Adaptation for Vision Transformers [45.3460867776953]
  We study parameter-efficient model adaptation strategies for vision transformers on the image classification task. We propose a parameter-efficient model adaptation framework, which first selects submodules by measuring local intrinsic dimensions. Our method performs the best in terms of the tradeoff between accuracy and parameter efficiency across 20 image classification datasets.
  arXiv  Detail & Related papers  (2022-03-29T05:30:09Z)
• Improving Task Adaptation for Cross-domain Few-shot Learning [41.821234589075445]
  Cross-domain few-shot classification aims to learn a classifier from previously unseen classes and domains with few labeled samples. We show that parametric adapters attached to convolutional layers with residual connections performs the best.
  arXiv  Detail & Related papers  (2021-07-01T10:47:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.