Evaluating the Values of Sources in Transfer Learning

• URL: http://arxiv.org/abs/2104.12567v1
• Date: Mon, 26 Apr 2021 13:35:24 GMT
• Title: Evaluating the Values of Sources in Transfer Learning
• Authors: Md Rizwan Parvez and Kai-Wei Chang
• Abstract summary: SEAL-Shap is an efficient source valuation framework for quantifying the usefulness of the sources. Our framework is not only effective in choosing useful transfer sources but also the source values match the intuitive source-target similarity.
• Score: 38.93955146476584
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Transfer learning that adapts a model trained on data-rich sources to low-resource targets has been widely applied in natural language processing (NLP). However, when training a transfer model over multiple sources, not every source is equally useful for the target. To better transfer a model, it is essential to understand the values of the sources. In this paper, we develop SEAL-Shap, an efficient source valuation framework for quantifying the usefulness of the sources (e.g., domains/languages) in transfer learning based on the Shapley value method. Experiments and comprehensive analyses on both cross-domain and cross-lingual transfers demonstrate that our framework is not only effective in choosing useful transfer sources but also the source values match the intuitive source-target similarity.

Related papers

• Learn from the Learnt: Source-Free Active Domain Adaptation via Contrastive Sampling and Visual Persistence [60.37934652213881]
  Domain Adaptation (DA) facilitates knowledge transfer from a source domain to a related target domain. This paper investigates a practical DA paradigm, namely Source data-Free Active Domain Adaptation (SFADA), where source data becomes inaccessible during adaptation. We present learn from the learnt (LFTL), a novel paradigm for SFADA to leverage the learnt knowledge from the source pretrained model and actively iterated models without extra overhead.
  arXiv  Detail & Related papers  (2024-07-26T17:51:58Z)
• Source-Free Domain Adaptation Guided by Vision and Vision-Language Pre-Training [23.56208527227504]
  Source-free domain adaptation (SFDA) aims to adapt a source model trained on a fully-labeled source domain to a related but unlabeled target domain. In the conventional SFDA pipeline, a large data (e.g. ImageNet) pre-trained feature extractor is used to initialize the source model. We introduce an integrated framework to incorporate pre-trained networks into the target adaptation process.
  arXiv  Detail & Related papers  (2024-05-05T14:48:13Z)
• Rethinking the Role of Pre-Trained Networks in Source-Free Domain Adaptation [26.481422574715126]
  Source-free domain adaptation (SFDA) aims to adapt a source model trained on a fully-labeled source domain to an unlabeled target domain. Large-data pre-trained networks are used to initialize source models during source training, and subsequently discarded. We propose to integrate the pre-trained network into the target adaptation process as it has diversified features important for generalization.
  arXiv  Detail & Related papers  (2022-12-15T02:25:22Z)
• Deep Transfer Learning for Multi-source Entity Linkage via Domain Adaptation [63.24594955429465]
  Multi-source entity linkage is critical in high-impact applications such as data cleaning and user stitching. AdaMEL is a deep transfer learning framework that learns generic high-level knowledge to perform multi-source entity linkage. Our framework achieves state-of-the-art results with 8.21% improvement on average over methods based on supervised learning.
  arXiv  Detail & Related papers  (2021-10-27T15:20:41Z)
• To Share or not to Share: Predicting Sets of Sources for Model Transfer Learning [22.846469609263416]
  We study the effects of model transfer on sequence labeling across various domains and tasks. Our methods based on model similarity and support vector machines are able to predict promising sources, resulting in performance increases of up to 24 F1 points.
  arXiv  Detail & Related papers  (2021-04-16T12:44:40Z)
• Unsupervised Multi-source Domain Adaptation Without Access to Source Data [58.551861130011886]
  Unsupervised Domain Adaptation (UDA) aims to learn a predictor model for an unlabeled domain by transferring knowledge from a separate labeled source domain. We propose a novel and efficient algorithm which automatically combines the source models with suitable weights in such a way that it performs at least as good as the best source model.
  arXiv  Detail & Related papers  (2021-04-05T10:45:12Z)
• Meta-learning Transferable Representations with a Single Target Domain [46.83481356352768]
  Fine-tuning and joint training do not always improve accuracy on downstream tasks. We propose Meta Representation Learning (MeRLin) to learn transferable features. MeRLin empirically outperforms previous state-of-the-art transfer learning algorithms on various real-world vision and NLP transfer learning benchmarks.
  arXiv  Detail & Related papers  (2020-11-03T01:57:37Z)
• Multi-source Attention for Unsupervised Domain Adaptation [15.900069711477542]
  We model source-selection as an attention-learning problem, where we learn attention over sources for a given target instance. For this purpose, we first independently learn source-specific classification models, and a relatedness map between sources and target domains. We then learn attention-weights over the sources for aggregating the predictions of the source-specific models.
  arXiv  Detail & Related papers  (2020-04-14T15:51:02Z)
• Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation [102.67010690592011]
  Unsupervised adaptationUDA (UDA) aims to leverage the knowledge learned from a labeled source dataset to solve similar tasks in a new unlabeled domain. Prior UDA methods typically require to access the source data when learning to adapt the model. This work tackles a practical setting where only a trained source model is available and how we can effectively utilize such a model without source data to solve UDA problems.
  arXiv  Detail & Related papers  (2020-02-20T03:13:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.