A Two-Stage Framework with Self-Supervised Distillation For Cross-Domain Text Classification

• URL: http://arxiv.org/abs/2304.09820v2
• Date: Wed, 10 Apr 2024 14:03:01 GMT
• Title: A Two-Stage Framework with Self-Supervised Distillation For Cross-Domain Text Classification
• Authors: Yunlong Feng, Bohan Li, Libo Qin, Xiao Xu, Wanxiang Che,
• Abstract summary: Cross-domain text classification aims to adapt models to a target domain that lacks labeled data. We propose a two-stage framework for cross-domain text classification.
• Score: 46.47734465505251
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Cross-domain text classification aims to adapt models to a target domain that lacks labeled data. It leverages or reuses rich labeled data from the different but related source domain(s) and unlabeled data from the target domain. To this end, previous work focuses on either extracting domain-invariant features or task-agnostic features, ignoring domain-aware features that may be present in the target domain and could be useful for the downstream task. In this paper, we propose a two-stage framework for cross-domain text classification. In the first stage, we finetune the model with mask language modeling (MLM) and labeled data from the source domain. In the second stage, we further fine-tune the model with self-supervised distillation (SSD) and unlabeled data from the target domain. We evaluate its performance on a public cross-domain text classification benchmark and the experiment results show that our method achieves new state-of-the-art results for both single-source domain adaptations (94.17% $\uparrow$1.03%) and multi-source domain adaptations (95.09% $\uparrow$1.34%).

Related papers

• Depth $F_1$: Improving Evaluation of Cross-Domain Text Classification by Measuring Semantic Generalizability [0.9954382983583578]
  Recent evaluations of cross-domain text classification models aim to measure the ability of a model to obtain domain-invariant performance in a target domain given labeled samples in a source domain. This evaluation strategy fails to account for the similarity between source and target domains, and may mask when models fail to transfer learning to specific target samples which are highly dissimilar from the source domain. We introduce Depth $F_1$, a novel cross-domain text classification performance metric.
  arXiv  Detail & Related papers  (2024-06-20T19:35:17Z)
• WUDA: Unsupervised Domain Adaptation Based on Weak Source Domain Labels [5.718326013810649]
  Unsupervised domain adaptation (UDA) for semantic segmentation addresses the cross-domain problem with fine source domain labels. This paper defines a new task: unsupervised domain adaptation based on weak source domain labels.
  arXiv  Detail & Related papers  (2022-10-05T08:28:57Z)
• Multi-source Few-shot Domain Adaptation [26.725145982321287]
  Multi-source Domain Adaptation (MDA) aims to transfer predictive models from multiple, fully-labeled source domains to an unlabeled target domain. In this paper, we investigate Multi-source Few-shot Domain Adaptation (MFDA), a new domain adaptation scenario with limited multi-source labels and unlabeled target data. We propose a novel framework, termed Multi-Source Few-shot Adaptation Network (MSFAN), which can be trained end-to-end in a non-adversarial manner.
  arXiv  Detail & Related papers  (2021-09-25T15:54:01Z)
• Cross-domain Contrastive Learning for Unsupervised Domain Adaptation [108.63914324182984]
  Unsupervised domain adaptation (UDA) aims to transfer knowledge learned from a fully-labeled source domain to a different unlabeled target domain. We build upon contrastive self-supervised learning to align features so as to reduce the domain discrepancy between training and testing sets.
  arXiv  Detail & Related papers  (2021-06-10T06:32:30Z)
• Prototypical Cross-domain Self-supervised Learning for Few-shot Unsupervised Domain Adaptation [91.58443042554903]
  We propose an end-to-end Prototypical Cross-domain Self-Supervised Learning (PCS) framework for Few-shot Unsupervised Domain Adaptation (FUDA) PCS not only performs cross-domain low-level feature alignment, but it also encodes and aligns semantic structures in the shared embedding space across domains. Compared with state-of-the-art methods, PCS improves the mean classification accuracy over different domain pairs on FUDA by 10.5%, 3.5%, 9.0%, and 13.2% on Office, Office-Home, VisDA-2017, and DomainNet, respectively.
  arXiv  Detail & Related papers  (2021-03-31T02:07:42Z)
• Inferring Latent Domains for Unsupervised Deep Domain Adaptation [54.963823285456925]
  Unsupervised Domain Adaptation (UDA) refers to the problem of learning a model in a target domain where labeled data are not available. This paper introduces a novel deep architecture which addresses the problem of UDA by automatically discovering latent domains in visual datasets. We evaluate our approach on publicly available benchmarks, showing that it outperforms state-of-the-art domain adaptation methods.
  arXiv  Detail & Related papers  (2021-03-25T14:33:33Z)
• Cross-domain Self-supervised Learning for Domain Adaptation with Few Source Labels [78.95901454696158]
  We propose a novel Cross-Domain Self-supervised learning approach for domain adaptation. Our method significantly boosts performance of target accuracy in the new target domain with few source labels.
  arXiv  Detail & Related papers  (2020-03-18T15:11:07Z)
• Deep Domain-Adversarial Image Generation for Domain Generalisation [115.21519842245752]
  Machine learning models typically suffer from the domain shift problem when trained on a source dataset and evaluated on a target dataset of different distribution. To overcome this problem, domain generalisation (DG) methods aim to leverage data from multiple source domains so that a trained model can generalise to unseen domains. We propose a novel DG approach based on emphDeep Domain-Adversarial Image Generation (DDAIG)
  arXiv  Detail & Related papers  (2020-03-12T23:17:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.