Leveraging Transfer Learning for Reliable Intelligence Identification on Vietnamese SNSs (ReINTEL)

• URL: http://arxiv.org/abs/2012.07557v2
• Date: Wed, 16 Dec 2020 15:10:07 GMT
• Title: Leveraging Transfer Learning for Reliable Intelligence Identification on Vietnamese SNSs (ReINTEL)
• Authors: Trung-Hieu Tran, Long Phan, Truong-Son Nguyen, Tien-Huy Nguyen
• Abstract summary: We exploit both of monolingual and multilingual pre-trained models. Our team achieved a score of 0.9378 at ROC-AUC metric in the private test set.
• Score: 0.8602553195689513
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper proposed several transformer-based approaches for Reliable Intelligence Identification on Vietnamese social network sites at VLSP 2020 evaluation campaign. We exploit both of monolingual and multilingual pre-trained models. Besides, we utilize the ensemble method to improve the robustness of different approaches. Our team achieved a score of 0.9378 at ROC-AUC metric in the private test set which is competitive to other participants.

Related papers

• Learning for Amalgamation: A Multi-Source Transfer Learning Framework For Sentiment Classification [1.9249287163937971]
  Our work explores beyond the common practice of transfer learning with a single pre-trained model. We propose LIFA, a framework to learn a unified embedding from several pre-trained models. We construct the first large-scale Vietnamese sentiment classification database.
  arXiv  Detail & Related papers  (2023-03-16T07:02:03Z)
• From English to More Languages: Parameter-Efficient Model Reprogramming for Cross-Lingual Speech Recognition [50.93943755401025]
  We propose a new parameter-efficient learning framework based on neural model reprogramming for cross-lingual speech recognition. We design different auxiliary neural architectures focusing on learnable pre-trained feature enhancement. Our methods outperform existing ASR tuning architectures and their extension with self-supervised losses.
  arXiv  Detail & Related papers  (2023-01-19T02:37:56Z)
• ConNER: Consistency Training for Cross-lingual Named Entity Recognition [96.84391089120847]
  Cross-lingual named entity recognition suffers from data scarcity in the target languages. We propose ConNER as a novel consistency training framework for cross-lingual NER.
  arXiv  Detail & Related papers  (2022-11-17T07:57:54Z)
• Curriculum Learning for Safe Mapless Navigation [71.55718344087657]
  This work investigates the effects of Curriculum Learning (CL)-based approaches on the agent's performance. In particular, we focus on the safety aspect of robotic mapless navigation, comparing over a standard end-to-end (E2E) training strategy.
  arXiv  Detail & Related papers  (2021-12-23T12:30:36Z)
• CoSSL: Co-Learning of Representation and Classifier for Imbalanced Semi-Supervised Learning [98.89092930354273]
  We propose a novel co-learning framework (CoSSL) with decoupled representation learning and classifier learning for imbalanced SSL. To handle the data imbalance, we devise Tail-class Feature Enhancement (TFE) for classifier learning. In experiments, we show that our approach outperforms other methods over a large range of shifted distributions.
  arXiv  Detail & Related papers  (2021-12-08T20:13:13Z)
• Private Language Model Adaptation for Speech Recognition [15.726921748859393]
  Speech model adaptation is crucial to handle the discrepancy between server-side proxy training data and actual data received on users' local devices. We introduce an efficient approach on continuously adapting neural network language models (NNLMs) on private devices with applications on automatic speech recognition.
  arXiv  Detail & Related papers  (2021-09-28T00:15:43Z)
• Boosting Cross-Lingual Transfer via Self-Learning with Uncertainty Estimation [34.97086123805344]
  Recent multilingual pre-trained language models have achieved remarkable zero-shot performance. We propose a self-learning framework that further utilizes unlabeled data of target languages. We evaluate our framework with uncertainties on two cross-lingual tasks including Named Entity Recognition (NER) and Natural Language Inference (NLI) covering 40 languages in total.
  arXiv  Detail & Related papers  (2021-09-01T05:26:46Z)
• AmericasNLI: Evaluating Zero-shot Natural Language Understanding of Pretrained Multilingual Models in Truly Low-resource Languages [75.08199398141744]
  We present AmericasNLI, an extension of XNLI (Conneau et al.), to 10 indigenous languages of the Americas. We conduct experiments with XLM-R, testing multiple zero-shot and translation-based approaches. We find that XLM-R's zero-shot performance is poor for all 10 languages, with an average performance of 38.62%.
  arXiv  Detail & Related papers  (2021-04-18T05:32:28Z)
• ReINTEL Challenge 2020: Exploiting Transfer Learning Models for Reliable Intelligence Identification on Vietnamese Social Network Sites [0.38073142980733]
  This paper presents the system that we propose for the Reliable Intelligence Indentification on Vietnamese Social Network Sites (ReINTEL) In this task, the VLSP 2020 provides a dataset with approximately 6,000 trainning news/posts annotated with reliable or unreliable labels, and a test set consists of 2,000 examples without labels. In our experiments, we achieve the AUC score of 94.52% on the private test set from ReINTEL's organizers.
  arXiv  Detail & Related papers  (2021-02-22T06:17:33Z)
• NLPBK at VLSP-2020 shared task: Compose transformer pretrained models for Reliable Intelligence Identification on Social network [0.0]
  This paper describes our method for tuning a transformer-based pretrained model, to adaptation with Reliable Intelligence Identification on Vietnamese SNSs problem. We also proposed a model that combines bert-base pretrained models with some metadata features, such as the number of comments, number of likes, images of SNS documents. With appropriate training techniques, our model is able to achieve 0.9392 ROC-AUC on public test set and the final version settles at top 2 ROC-AUC (0.9513) on private test set.
  arXiv  Detail & Related papers  (2021-01-29T16:19:28Z)
• Cross-lingual Retrieval for Iterative Self-Supervised Training [66.3329263451598]
  Cross-lingual alignment can be further improved by training seq2seq models on sentence pairs mined using their own encoder outputs. We develop a new approach -- cross-lingual retrieval for iterative self-supervised training.
  arXiv  Detail & Related papers  (2020-06-16T21:30:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.