Self-training for Few-shot Transfer Across Extreme Task Differences
- URL: http://arxiv.org/abs/2010.07734v2
- Date: Wed, 17 Mar 2021 16:11:57 GMT
- Title: Self-training for Few-shot Transfer Across Extreme Task Differences
- Authors: Cheng Perng Phoo, Bharath Hariharan
- Abstract summary: Most few-shot learning techniques are pre-trained on a large, labeled "base dataset"
In problem domains where such large labeled datasets are not available for pre-training, one must resort to pre-training in a different "source" problem domain.
Traditional few-shot and transfer learning techniques fail in the presence of such extreme differences between the source and target tasks.
- Score: 46.07212902030414
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Most few-shot learning techniques are pre-trained on a large, labeled "base
dataset". In problem domains where such large labeled datasets are not
available for pre-training (e.g., X-ray, satellite images), one must resort to
pre-training in a different "source" problem domain (e.g., ImageNet), which can
be very different from the desired target task. Traditional few-shot and
transfer learning techniques fail in the presence of such extreme differences
between the source and target tasks. In this paper, we present a simple and
effective solution to tackle this extreme domain gap: self-training a source
domain representation on unlabeled data from the target domain. We show that
this improves one-shot performance on the target domain by 2.9 points on
average on the challenging BSCD-FSL benchmark consisting of datasets from
multiple domains. Our code is available at https://github.com/cpphoo/STARTUP.
Related papers
- CDFSL-V: Cross-Domain Few-Shot Learning for Videos [58.37446811360741]
Few-shot video action recognition is an effective approach to recognizing new categories with only a few labeled examples.
Existing methods in video action recognition rely on large labeled datasets from the same domain.
We propose a novel cross-domain few-shot video action recognition method that leverages self-supervised learning and curriculum learning.
arXiv Detail & Related papers (2023-09-07T19:44:27Z) - Understanding Cross-Domain Few-Shot Learning: An Experimental Study [17.81177649496765]
Cross-domain few-shot learning has drawn increasing attention for handling large differences between the source and target domains.
Recent works have considered exploiting small-scale unlabeled data from the target domain during the pre-training stage.
This data enables self-supervised pre-training on the target domain, in addition to supervised pre-training on the source domain.
arXiv Detail & Related papers (2022-02-01T12:35:25Z) - Prior Omission of Dissimilar Source Domain(s) for Cost-Effective
Few-Shot Learning [24.647313693814798]
Few-shot slot tagging is an emerging research topic in the field of Natural Language Understanding (NLU)
With sufficient annotated data from source domains, the key challenge is how to train and adapt the model to another target domain which only has few labels.
arXiv Detail & Related papers (2021-09-11T09:30:59Z) - Multilevel Knowledge Transfer for Cross-Domain Object Detection [26.105283273950942]
Domain shift is a well known problem where a model trained on a particular domain (source) does not perform well when exposed to samples from a different domain (target)
In this work, we address the domain shift problem for the object detection task.
Our approach relies on gradually removing the domain shift between the source and the target domains.
arXiv Detail & Related papers (2021-08-02T15:24:40Z) - Dynamic Distillation Network for Cross-Domain Few-Shot Recognition with
Unlabeled Data [21.348965677980104]
We tackle the problem of cross-domain few-shot recognition with unlabeled target data.
STARTUP was the first method that tackles this problem using self-training.
We propose a simple dynamic distillation-based approach to facilitate unlabeled images from the novel/base dataset.
arXiv Detail & Related papers (2021-06-14T23:44:34Z) - Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised
Pre-Training [67.71228426496013]
We show that using target domain data during pre-training leads to large performance improvements across a variety of setups.
We find that pre-training on multiple domains improves performance generalization on domains not seen during training.
arXiv Detail & Related papers (2021-04-02T12:53:15Z) - Surprisingly Simple Semi-Supervised Domain Adaptation with Pretraining
and Consistency [93.89773386634717]
Visual domain adaptation involves learning to classify images from a target visual domain using labels available in a different source domain.
We show that in the presence of a few target labels, simple techniques like self-supervision (via rotation prediction) and consistency regularization can be effective without any adversarial alignment to learn a good target classifier.
Our Pretraining and Consistency (PAC) approach, can achieve state of the art accuracy on this semi-supervised domain adaptation task, surpassing multiple adversarial domain alignment methods, across multiple datasets.
arXiv Detail & Related papers (2021-01-29T18:40:17Z) - Cross-domain few-shot learning with unlabelled data [1.2183405753834562]
Few shot learning aims to solve the data scarcity problem.
We propose a new setting some unlabelled data from the target domain is provided.
We come up with a self-supervised learning method to fully utilize the knowledge in the labeled training set and the unlabelled set.
arXiv Detail & Related papers (2021-01-19T23:41:57Z) - A Review of Single-Source Deep Unsupervised Visual Domain Adaptation [81.07994783143533]
Large-scale labeled training datasets have enabled deep neural networks to excel across a wide range of benchmark vision tasks.
In many applications, it is prohibitively expensive and time-consuming to obtain large quantities of labeled data.
To cope with limited labeled training data, many have attempted to directly apply models trained on a large-scale labeled source domain to another sparsely labeled or unlabeled target domain.
arXiv Detail & Related papers (2020-09-01T00:06:50Z) - Deep Domain-Adversarial Image Generation for Domain Generalisation [115.21519842245752]
Machine learning models typically suffer from the domain shift problem when trained on a source dataset and evaluated on a target dataset of different distribution.
To overcome this problem, domain generalisation (DG) methods aim to leverage data from multiple source domains so that a trained model can generalise to unseen domains.
We propose a novel DG approach based on emphDeep Domain-Adversarial Image Generation (DDAIG)
arXiv Detail & Related papers (2020-03-12T23:17:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.