Cross-Domain Sentiment Classification with Contrastive Learning and Mutual Information Maximization

• URL: http://arxiv.org/abs/2010.16088v2
• Date: Thu, 12 Nov 2020 02:52:20 GMT
• Title: Cross-Domain Sentiment Classification with Contrastive Learning and Mutual Information Maximization
• Authors: Tian Li, Xiang Chen, Shanghang Zhang, Zhen Dong, Kurt Keutzer
• Abstract summary: We propose CLIM: Contrastive Learning with mutual Information Maximization, to explore the potential of CL on cross-domain sentiment classification. Due to scarcity of labels on the target domain, we introduce mutual information (MIM) apart from CL to exploit the features that best support the final prediction. We achieve new state-of-the-art results on the Amazon-review dataset as well as the airlines dataset, showing the efficacy of our proposed method CLIM.
• Score: 48.41392004071199
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Contrastive learning (CL) has been successful as a powerful representation learning method. In this work we propose CLIM: Contrastive Learning with mutual Information Maximization, to explore the potential of CL on cross-domain sentiment classification. To the best of our knowledge, CLIM is the first to adopt contrastive learning for natural language processing (NLP) tasks across domains. Due to scarcity of labels on the target domain, we introduce mutual information maximization (MIM) apart from CL to exploit the features that best support the final prediction. Furthermore, MIM is able to maintain a relatively balanced distribution of the model's prediction, and enlarges the margin between classes on the target domain. The larger margin increases our model's robustness and enables the same classifier to be optimal across domains. Consequently, we achieve new state-of-the-art results on the Amazon-review dataset as well as the airlines dataset, showing the efficacy of our proposed method CLIM.

Related papers

• CLLMFS: A Contrastive Learning enhanced Large Language Model Framework for Few-Shot Named Entity Recognition [3.695767900907561]
  CLLMFS is a Contrastive Learning enhanced Large Language Model framework for Few-Shot Named Entity Recognition. It integrates Low-Rank Adaptation (LoRA) and contrastive learning mechanisms specifically tailored for few-shot NER. Our method has achieved state-of-the-art performance improvements on F1-score ranging from 2.58% to 97.74% over existing best-performing methods.
  arXiv  Detail & Related papers  (2024-08-23T04:44:05Z)
• Improving Intrusion Detection with Domain-Invariant Representation Learning in Latent Space [4.871119861180455]
  We introduce a two-phase representation learning technique using multi-task learning. We disentangle the latent space by minimizing the mutual information between the prior and latent space. We assess the model's efficacy across multiple cybersecurity datasets.
  arXiv  Detail & Related papers  (2023-12-28T17:24:13Z)
• Multi-level Consistency Learning for Semi-supervised Domain Adaptation [85.90600060675632]
  Semi-supervised domain adaptation (SSDA) aims to apply knowledge learned from a fully labeled source domain to a scarcely labeled target domain. We propose a Multi-level Consistency Learning framework for SSDA.
  arXiv  Detail & Related papers  (2022-05-09T06:41:18Z)
• Unified Instance and Knowledge Alignment Pretraining for Aspect-based Sentiment Analysis [96.53859361560505]
  Aspect-based Sentiment Analysis (ABSA) aims to determine the sentiment polarity towards an aspect. There always exists severe domain shift between the pretraining and downstream ABSA datasets. We introduce a unified alignment pretraining framework into the vanilla pretrain-finetune pipeline.
  arXiv  Detail & Related papers  (2021-10-26T04:03:45Z)
• Unsupervised Domain Adaptation for Semantic Segmentation via Low-level Edge Information Transfer [27.64947077788111]
  Unsupervised domain adaptation for semantic segmentation aims to make models trained on synthetic data adapt to real images. Previous feature-level adversarial learning methods only consider adapting models on the high-level semantic features. We present the first attempt at explicitly using low-level edge information, which has a small inter-domain gap, to guide the transfer of semantic information.
  arXiv  Detail & Related papers  (2021-09-18T11:51:31Z)
• Source-Free Open Compound Domain Adaptation in Semantic Segmentation [99.82890571842603]
  In SF-OCDA, only the source pre-trained model and the target data are available to learn the target model. We propose the Cross-Patch Style Swap (CPSS) to diversify samples with various patch styles in the feature-level. Our method produces state-of-the-art results on the C-Driving dataset.
  arXiv  Detail & Related papers  (2021-06-07T08:38:41Z)
• Dual-Teacher++: Exploiting Intra-domain and Inter-domain Knowledge with Reliable Transfer for Cardiac Segmentation [69.09432302497116]
  We propose a cutting-edge semi-supervised domain adaptation framework, namely Dual-Teacher++. We design novel dual teacher models, including an inter-domain teacher model to explore cross-modality priors from source domain (e.g., MR) and an intra-domain teacher model to investigate the knowledge beneath unlabeled target domain. In this way, the student model can obtain reliable dual-domain knowledge and yield improved performance on target domain data.
  arXiv  Detail & Related papers  (2021-01-07T05:17:38Z)
• Cluster, Split, Fuse, and Update: Meta-Learning for Open Compound Domain Adaptive Semantic Segmentation [102.42638795864178]
  We propose a principled meta-learning based approach to OCDA for semantic segmentation. We cluster target domain into multiple sub-target domains by image styles, extracted in an unsupervised manner. A meta-learner is thereafter deployed to learn to fuse sub-target domain-specific predictions, conditioned upon the style code. We learn to online update the model by model-agnostic meta-learning (MAML) algorithm, thus to further improve generalization.
  arXiv  Detail & Related papers  (2020-12-15T13:21:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.