Learning for Amalgamation: A Multi-Source Transfer Learning Framework For Sentiment Classification

• URL: http://arxiv.org/abs/2303.09115v1
• Date: Thu, 16 Mar 2023 07:02:03 GMT
• Title: Learning for Amalgamation: A Multi-Source Transfer Learning Framework For Sentiment Classification
• Authors: Cuong V. Nguyen, Khiem H. Le, Anh M. Tran, Quang H. Pham, Binh T. Nguyen
• Abstract summary: Our work explores beyond the common practice of transfer learning with a single pre-trained model. We propose LIFA, a framework to learn a unified embedding from several pre-trained models. We construct the first large-scale Vietnamese sentiment classification database.
• Score: 1.9249287163937971
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Transfer learning plays an essential role in Deep Learning, which can remarkably improve the performance of the target domain, whose training data is not sufficient. Our work explores beyond the common practice of transfer learning with a single pre-trained model. We focus on the task of Vietnamese sentiment classification and propose LIFA, a framework to learn a unified embedding from several pre-trained models. We further propose two more LIFA variants that encourage the pre-trained models to either cooperate or compete with one another. Studying these variants sheds light on the success of LIFA by showing that sharing knowledge among the models is more beneficial for transfer learning. Moreover, we construct the AISIA-VN-Review-F dataset, the first large-scale Vietnamese sentiment classification database. We conduct extensive experiments on the AISIA-VN-Review-F and existing benchmarks to demonstrate the efficacy of LIFA compared to other techniques. To contribute to the Vietnamese NLP research, we publish our source code and datasets to the research community upon acceptance.

Related papers

• KBAlign: Efficient Self Adaptation on Specific Knowledge Bases [73.34893326181046]
  Large language models (LLMs) usually rely on retrieval-augmented generation to exploit knowledge materials in an instant manner. We propose KBAlign, an approach designed for efficient adaptation to downstream tasks involving knowledge bases. Our method utilizes iterative training with self-annotated data such as Q&A pairs and revision suggestions, enabling the model to grasp the knowledge content efficiently.
  arXiv  Detail & Related papers  (2024-11-22T08:21:03Z)
• An Active Learning Framework for Inclusive Generation by Large Language Models [32.16984263644299]
  Large Language Models (LLMs) generate text representative of diverse sub-populations. We propose a novel clustering-based active learning framework, enhanced with knowledge distillation. We construct two new datasets in tandem with model training, showing a performance improvement of 2%-10% over baseline models.
  arXiv  Detail & Related papers  (2024-10-17T15:09:35Z)
• High-Performance Few-Shot Segmentation with Foundation Models: An Empirical Study [64.06777376676513]
  We develop a few-shot segmentation (FSS) framework based on foundation models. To be specific, we propose a simple approach to extract implicit knowledge from foundation models to construct coarse correspondence. Experiments on two widely used datasets demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-09-10T08:04:11Z)
• Accelerating Large Language Model Pretraining via LFR Pedagogy: Learn, Focus, and Review [50.78587571704713]
  Learn-Focus-Review (LFR) is a dynamic training approach that adapts to the model's learning progress. LFR tracks the model's learning performance across data blocks (sequences of tokens) and prioritizes revisiting challenging regions of the dataset. Compared to baseline models trained on the full datasets, LFR consistently achieved lower perplexity and higher accuracy.
  arXiv  Detail & Related papers  (2024-09-10T00:59:18Z)
• UNIDEAL: Curriculum Knowledge Distillation Federated Learning [17.817181326740698]
  Federated Learning (FL) has emerged as a promising approach to enable collaborative learning among multiple clients. In this paper, we present UNI, a novel FL algorithm specifically designed to tackle the challenges of cross-domain scenarios. Our results demonstrate that UNI achieves superior performance in terms of both model accuracy and communication efficiency.
  arXiv  Detail & Related papers  (2023-09-16T11:30:29Z)
• Collaborating Heterogeneous Natural Language Processing Tasks via Federated Learning [55.99444047920231]
  The proposed ATC framework achieves significant improvements compared with various baseline methods. We conduct extensive experiments on six widely-used datasets covering both Natural Language Understanding (NLU) and Natural Language Generation (NLG) tasks.
  arXiv  Detail & Related papers  (2022-12-12T09:27:50Z)
• Leveraging Different Learning Styles for Improved Knowledge Distillation in Biomedical Imaging [0.9208007322096533]
  Our work endeavors to leverage the concept of knowledge diversification to improve the performance of model compression techniques like Knowledge Distillation (KD) and Mutual Learning (ML) We use a single-teacher and two-student network in a unified framework that not only allows for the transfer of knowledge from teacher to students (KD) but also encourages collaborative learning between students (ML) Unlike the conventional approach, where the teacher shares the same knowledge in the form of predictions or feature representations with the student network, our proposed approach employs a more diversified strategy by training one student with predictions and the other with feature maps from the teacher.
  arXiv  Detail & Related papers  (2022-12-06T12:40:45Z)
• Leveraging Transfer Learning for Reliable Intelligence Identification on Vietnamese SNSs (ReINTEL) [0.8602553195689513]
  We exploit both of monolingual and multilingual pre-trained models. Our team achieved a score of 0.9378 at ROC-AUC metric in the private test set.
  arXiv  Detail & Related papers  (2020-12-10T15:43:50Z)
• Self-supervised Co-training for Video Representation Learning [103.69904379356413]
  We investigate the benefit of adding semantic-class positives to instance-based Info Noise Contrastive Estimation training. We propose a novel self-supervised co-training scheme to improve the popular infoNCE loss. We evaluate the quality of the learnt representation on two different downstream tasks: action recognition and video retrieval.
  arXiv  Detail & Related papers  (2020-10-19T17:59:01Z)
• Learning From Multiple Experts: Self-paced Knowledge Distillation for Long-tailed Classification [106.08067870620218]
  We propose a self-paced knowledge distillation framework, termed Learning From Multiple Experts (LFME) We refer to these models as 'Experts', and the proposed LFME framework aggregates the knowledge from multiple 'Experts' to learn a unified student model. We conduct extensive experiments and demonstrate that our method is able to achieve superior performances compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-01-06T12:57:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.