Few-Shot Learning as Domain Adaptation: Algorithm and Analysis
- URL: http://arxiv.org/abs/2002.02050v3
- Date: Mon, 27 Jul 2020 06:26:34 GMT
- Title: Few-Shot Learning as Domain Adaptation: Algorithm and Analysis
- Authors: Jiechao Guan, Zhiwu Lu, Tao Xiang, Ji-Rong Wen
- Abstract summary: Few-shot learning uses prior knowledge learned from the seen classes to recognize the unseen classes.
This class-difference-caused distribution shift can be considered as a special case of domain shift.
We propose a prototypical domain adaptation network with attention (DAPNA) to explicitly tackle such a domain shift problem in a meta-learning framework.
- Score: 120.75020271706978
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: To recognize the unseen classes with only few samples, few-shot learning
(FSL) uses prior knowledge learned from the seen classes. A major challenge for
FSL is that the distribution of the unseen classes is different from that of
those seen, resulting in poor generalization even when a model is meta-trained
on the seen classes. This class-difference-caused distribution shift can be
considered as a special case of domain shift. In this paper, for the first
time, we propose a domain adaptation prototypical network with attention
(DAPNA) to explicitly tackle such a domain shift problem in a meta-learning
framework. Specifically, armed with a set transformer based attention module,
we construct each episode with two sub-episodes without class overlap on the
seen classes to simulate the domain shift between the seen and unseen classes.
To align the feature distributions of the two sub-episodes with limited
training samples, a feature transfer network is employed together with a margin
disparity discrepancy (MDD) loss. Importantly, theoretical analysis is provided
to give the learning bound of our DAPNA. Extensive experiments show that our
DAPNA outperforms the state-of-the-art FSL alternatives, often by significant
margins.
Related papers
- SALUDA: Surface-based Automotive Lidar Unsupervised Domain Adaptation [62.889835139583965]
We introduce an unsupervised auxiliary task of learning an implicit underlying surface representation simultaneously on source and target data.
As both domains share the same latent representation, the model is forced to accommodate discrepancies between the two sources of data.
Our experiments demonstrate that our method achieves a better performance than the current state of the art, both in real-to-real and synthetic-to-real scenarios.
arXiv Detail & Related papers (2023-04-06T17:36:23Z) - From Big to Small: Adaptive Learning to Partial-Set Domains [94.92635970450578]
Domain adaptation targets at knowledge acquisition and dissemination from a labeled source domain to an unlabeled target domain under distribution shift.
Recent advances show that deep pre-trained models of large scale endow rich knowledge to tackle diverse downstream tasks of small scale.
This paper introduces Partial Domain Adaptation (PDA), a learning paradigm that relaxes the identical class space assumption to that the source class space subsumes the target class space.
arXiv Detail & Related papers (2022-03-14T07:02:45Z) - Few-Shot Classification in Unseen Domains by Episodic Meta-Learning
Across Visual Domains [36.98387822136687]
Few-shot classification aims to carry out classification given only few labeled examples for the categories of interest.
In this paper, we present a unique learning framework for domain-generalized few-shot classification.
By advancing meta-learning strategies, our learning framework exploits data across multiple source domains to capture domain-invariant features.
arXiv Detail & Related papers (2021-12-27T06:54:11Z) - Transferrable Contrastive Learning for Visual Domain Adaptation [108.98041306507372]
Transferrable Contrastive Learning (TCL) is a self-supervised learning paradigm tailored for domain adaptation.
TCL penalizes cross-domain intra-class domain discrepancy between source and target through a clean and novel contrastive loss.
The free lunch is, thanks to the incorporation of contrastive learning, TCL relies on a moving-averaged key encoder that naturally achieves a temporally ensembled version of pseudo labels for target data.
arXiv Detail & Related papers (2021-12-14T16:23:01Z) - CADA: Multi-scale Collaborative Adversarial Domain Adaptation for
Unsupervised Optic Disc and Cup Segmentation [3.587294308501889]
We propose a novel unsupervised domain adaptation framework, called Collaborative Adrial Domain Adaptation (CADA)
Our proposed CADA is an interactive paradigm that presents an exquisite collaborative adaptation through both adversarial learning and ensembling weights at different network layers.
We show that our CADA model incorporating multi-scale input training can overcome performance degradation and outperform state-of-the-art domain adaptation methods.
arXiv Detail & Related papers (2021-10-05T23:44:26Z) - Improving Transferability of Domain Adaptation Networks Through Domain
Alignment Layers [1.3766148734487902]
Multi-source unsupervised domain adaptation (MSDA) aims at learning a predictor for an unlabeled domain by assigning weak knowledge from a bag of source models.
We propose to embed Multi-Source version of DomaIn Alignment Layers (MS-DIAL) at different levels of the predictor.
Our approach can improve state-of-the-art MSDA methods, yielding relative gains of up to +30.64% on their classification accuracies.
arXiv Detail & Related papers (2021-09-06T18:41:19Z) - Revisiting Mid-Level Patterns for Cross-Domain Few-Shot Recognition [31.81367604846625]
Cross-domain few-shot learning is proposed to transfer knowledge from general-domain base classes to special-domain novel classes.
In this paper, we study a challenging subset of CDFSL where the novel classes are in distant domains from base classes.
We propose a residual-prediction task to encourage mid-level features to learn discriminative information of each sample.
arXiv Detail & Related papers (2020-08-07T12:45:39Z) - Adaptive Risk Minimization: Learning to Adapt to Domain Shift [109.87561509436016]
A fundamental assumption of most machine learning algorithms is that the training and test data are drawn from the same underlying distribution.
In this work, we consider the problem setting of domain generalization, where the training data are structured into domains and there may be multiple test time shifts.
We introduce the framework of adaptive risk minimization (ARM), in which models are directly optimized for effective adaptation to shift by learning to adapt on the training domains.
arXiv Detail & Related papers (2020-07-06T17:59:30Z) - Domain-Adaptive Few-Shot Learning [124.51420562201407]
We propose a novel domain-adversarial network (DAPN) model for domain-adaptive few-shot learning.
Our solution is to explicitly enhance the source/target per-class separation before domain-adaptive feature embedding learning.
arXiv Detail & Related papers (2020-03-19T08:31:14Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.