Continuous Transfer Learning with Label-informed Distribution Alignment

• URL: http://arxiv.org/abs/2006.03230v1
• Date: Fri, 5 Jun 2020 04:44:58 GMT
• Title: Continuous Transfer Learning with Label-informed Distribution Alignment
• Authors: Jun Wu, Jingrui He
• Abstract summary: We study a novel continuous transfer learning setting with a time evolving target domain. One major challenge associated with continuous transfer learning is the potential occurrence of negative transfer. We propose a generic adversarial Variational Auto-encoder framework named TransLATE.
• Score: 42.34180707803632
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Transfer learning has been successfully applied across many high-impact applications. However, most existing work focuses on the static transfer learning setting, and very little is devoted to modeling the time evolving target domain, such as the online reviews for movies. To bridge this gap, in this paper, we study a novel continuous transfer learning setting with a time evolving target domain. One major challenge associated with continuous transfer learning is the potential occurrence of negative transfer as the target domain evolves over time. To address this challenge, we propose a novel label-informed C-divergence between the source and target domains in order to measure the shift of data distributions as well as to identify potential negative transfer. We then derive the error bound for the target domain using the empirical estimate of our proposed C-divergence. Furthermore, we propose a generic adversarial Variational Auto-encoder framework named TransLATE by minimizing the classification error and C-divergence of the target domain between consecutive time stamps in a latent feature space. In addition, we define a transfer signature for characterizing the negative transfer based on C-divergence, which indicates that larger C-divergence implies a higher probability of negative transfer in real scenarios. Extensive experiments on synthetic and real data sets demonstrate the effectiveness of our TransLATE framework.

Related papers

• Overcoming Negative Transfer by Online Selection: Distant Domain Adaptation for Fault Diagnosis [42.85741244467877]
  The term distant domain adaptation problem' describes the challenge of adapting from a labeled source domain to a significantly disparate unlabeled target domain. This problem exhibits the risk of negative transfer, where extraneous knowledge from the source domain adversely affects the target domain performance. In response to this challenge, we propose a novel Online Selective Adversarial Alignment (OSAA) approach.
  arXiv  Detail & Related papers  (2024-05-25T07:17:47Z)
• Enhancing Continuous Domain Adaptation with Multi-Path Transfer Curriculum [8.712743055816958]
  Continuous Domain Adaptation has emerged as an effective technique to close the distribution gap between training and testing data. This paper contributes a novel method, W-MPOT, which rigorously addresses the domain ordering and error accumulation problems. We extensively evaluate W-MPOT on multiple datasets, achieving up to 54.1% accuracy improvement on multi-session Alzheimer MR image classification and 94.7% MSE reduction on battery capacity estimation.
  arXiv  Detail & Related papers  (2024-02-26T15:59:38Z)
• Logit Margin Matters: Improving Transferable Targeted Adversarial Attack by Logit Calibration [85.71545080119026]
  Cross-Entropy (CE) loss function is insufficient to learn transferable targeted adversarial examples. We propose two simple and effective logit calibration methods, which are achieved by downscaling the logits with a temperature factor and an adaptive margin. Experiments conducted on the ImageNet dataset validate the effectiveness of the proposed methods.
  arXiv  Detail & Related papers  (2023-03-07T06:42:52Z)
• Can You Label Less by Using Out-of-Domain Data? Active & Transfer Learning with Few-shot Instructions [58.69255121795761]
  We propose a novel Active Transfer Few-shot Instructions (ATF) approach which requires no fine-tuning. ATF leverages the internal linguistic knowledge of pre-trained language models (PLMs) to facilitate the transfer of information. We show that annotation of just a few target-domain samples via active learning can be beneficial for transfer, but the impact diminishes with more annotation effort.
  arXiv  Detail & Related papers  (2022-11-21T19:03:31Z)
• Coarse to Fine: Domain Adaptive Crowd Counting via Adversarial Scoring Network [58.05473757538834]
  This paper proposes a novel adversarial scoring network (ASNet) to bridge the gap across domains from coarse to fine granularity. Three sets of migration experiments show that the proposed methods achieve state-of-the-art counting performance.
  arXiv  Detail & Related papers  (2021-07-27T14:47:24Z)
• Towards Accurate Knowledge Transfer via Target-awareness Representation Disentanglement [56.40587594647692]
  We propose a novel transfer learning algorithm, introducing the idea of Target-awareness REpresentation Disentanglement (TRED) TRED disentangles the relevant knowledge with respect to the target task from the original source model and used as a regularizer during fine-tuning the target model. Experiments on various real world datasets show that our method stably improves the standard fine-tuning by more than 2% in average.
  arXiv  Detail & Related papers  (2020-10-16T17:45:08Z)
• CSCL: Critical Semantic-Consistent Learning for Unsupervised Domain Adaptation [42.226842513334184]
  We develop a new Critical Semantic-Consistent Learning model, which mitigates the discrepancy of both domain-wise and category-wise distributions. Specifically, a critical transfer based adversarial framework is designed to highlight transferable domain-wise knowledge while neglecting untransferable knowledge.
  arXiv  Detail & Related papers  (2020-08-24T14:12:04Z)
• Unsupervised Transfer Learning with Self-Supervised Remedy [60.315835711438936]
  Generalising deep networks to novel domains without manual labels is challenging to deep learning. Pre-learned knowledge does not transfer well without making strong assumptions about the learned and the novel domains. In this work, we aim to learn a discriminative latent space of the unlabelled target data in a novel domain by knowledge transfer from labelled related domains.
  arXiv  Detail & Related papers  (2020-06-08T16:42:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.