To Share or not to Share: Predicting Sets of Sources for Model Transfer Learning

• URL: http://arxiv.org/abs/2104.08078v1
• Date: Fri, 16 Apr 2021 12:44:40 GMT
• Title: To Share or not to Share: Predicting Sets of Sources for Model Transfer Learning
• Authors: Lukas Lange, Jannik Str\"otgen, Heike Adel, Dietrich Klakow
• Abstract summary: We study the effects of model transfer on sequence labeling across various domains and tasks. Our methods based on model similarity and support vector machines are able to predict promising sources, resulting in performance increases of up to 24 F1 points.
• Score: 22.846469609263416
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In low-resource settings, model transfer can help to overcome a lack of labeled data for many tasks and domains. However, predicting useful transfer sources is a challenging problem, as even the most similar sources might lead to unexpected negative transfer results. Thus, ranking methods based on task and text similarity may not be sufficient to identify promising sources. To tackle this problem, we propose a method to automatically determine which and how many sources should be exploited. For this, we study the effects of model transfer on sequence labeling across various domains and tasks and show that our methods based on model similarity and support vector machines are able to predict promising sources, resulting in performance increases of up to 24 F1 points.

Related papers

• Balancing Discriminability and Transferability for Source-Free Domain Adaptation [55.143687986324935]
  Conventional domain adaptation (DA) techniques aim to improve domain transferability by learning domain-invariant representations. The requirement of simultaneous access to labeled source and unlabeled target renders them unsuitable for the challenging source-free DA setting. We derive novel insights to show that a mixup between original and corresponding translated generic samples enhances the discriminability-transferability trade-off.
  arXiv  Detail & Related papers  (2022-06-16T09:06:22Z)
• Source-Free Domain Adaptation via Distribution Estimation [106.48277721860036]
  Domain Adaptation aims to transfer the knowledge learned from a labeled source domain to an unlabeled target domain whose data distributions are different. Recently, Source-Free Domain Adaptation (SFDA) has drawn much attention, which tries to tackle domain adaptation problem without using source data. In this work, we propose a novel framework called SFDA-DE to address SFDA task via source Distribution Estimation.
  arXiv  Detail & Related papers  (2022-04-24T12:22:19Z)
• Source data selection for out-of-domain generalization [0.76146285961466]
  Poor selection of a source dataset can lead to poor performance on the target. We propose two source selection methods that are based on the multi-bandit theory and random search. Our proposals can be viewed as diagnostics for the existence of a reweighted source subsamples that perform better than the random selection of available samples.
  arXiv  Detail & Related papers  (2022-02-04T14:37:31Z)
• Source-Free Domain Adaptive Fundus Image Segmentation with Denoised Pseudo-Labeling [56.98020855107174]
  Domain adaptation typically requires to access source domain data to utilize their distribution information for domain alignment with the target data. In many real-world scenarios, the source data may not be accessible during the model adaptation in the target domain due to privacy issue. We present a novel denoised pseudo-labeling method for this problem, which effectively makes use of the source model and unlabeled target data.
  arXiv  Detail & Related papers  (2021-09-19T06:38:21Z)
• Evaluating the Values of Sources in Transfer Learning [38.93955146476584]
  SEAL-Shap is an efficient source valuation framework for quantifying the usefulness of the sources. Our framework is not only effective in choosing useful transfer sources but also the source values match the intuitive source-target similarity.
  arXiv  Detail & Related papers  (2021-04-26T13:35:24Z)
• Unsupervised Multi-source Domain Adaptation Without Access to Source Data [58.551861130011886]
  Unsupervised Domain Adaptation (UDA) aims to learn a predictor model for an unlabeled domain by transferring knowledge from a separate labeled source domain. We propose a novel and efficient algorithm which automatically combines the source models with suitable weights in such a way that it performs at least as good as the best source model.
  arXiv  Detail & Related papers  (2021-04-05T10:45:12Z)
• Unsupervised Domain Adaptation in the Absence of Source Data [0.7366405857677227]
  We propose an unsupervised method for adapting a source classifier to a target domain that varies from the source domain along natural axes. We validate our method in scenarios where the distribution shift involves brightness, contrast, and rotation and show that it outperforms fine-tuning baselines in scenarios with limited labeled data.
  arXiv  Detail & Related papers  (2020-07-20T16:22:14Z)
• Multi-source Attention for Unsupervised Domain Adaptation [15.900069711477542]
  We model source-selection as an attention-learning problem, where we learn attention over sources for a given target instance. For this purpose, we first independently learn source-specific classification models, and a relatedness map between sources and target domains. We then learn attention-weights over the sources for aggregating the predictions of the source-specific models.
  arXiv  Detail & Related papers  (2020-04-14T15:51:02Z)
• Universal Source-Free Domain Adaptation [57.37520645827318]
  We propose a novel two-stage learning process for domain adaptation. In the Procurement stage, we aim to equip the model for future source-free deployment, assuming no prior knowledge of the upcoming category-gap and domain-shift. In the Deployment stage, the goal is to design a unified adaptation algorithm capable of operating across a wide range of category-gaps.
  arXiv  Detail & Related papers  (2020-04-09T07:26:20Z)
• Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation [102.67010690592011]
  Unsupervised adaptationUDA (UDA) aims to leverage the knowledge learned from a labeled source dataset to solve similar tasks in a new unlabeled domain. Prior UDA methods typically require to access the source data when learning to adapt the model. This work tackles a practical setting where only a trained source model is available and how we can effectively utilize such a model without source data to solve UDA problems.
  arXiv  Detail & Related papers  (2020-02-20T03:13:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.