NLPBK at VLSP-2020 shared task: Compose transformer pretrained models for Reliable Intelligence Identification on Social network

• URL: http://arxiv.org/abs/2101.12672v1
• Date: Fri, 29 Jan 2021 16:19:28 GMT
• Title: NLPBK at VLSP-2020 shared task: Compose transformer pretrained models for Reliable Intelligence Identification on Social network
• Authors: Thanh Chinh Nguyen, Van Nha Nguyen
• Abstract summary: This paper describes our method for tuning a transformer-based pretrained model, to adaptation with Reliable Intelligence Identification on Vietnamese SNSs problem. We also proposed a model that combines bert-base pretrained models with some metadata features, such as the number of comments, number of likes, images of SNS documents. With appropriate training techniques, our model is able to achieve 0.9392 ROC-AUC on public test set and the final version settles at top 2 ROC-AUC (0.9513) on private test set.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper describes our method for tuning a transformer-based pretrained model, to adaptation with Reliable Intelligence Identification on Vietnamese SNSs problem. We also proposed a model that combines bert-base pretrained models with some metadata features, such as the number of comments, number of likes, images of SNS documents,... to improved results for VLSP shared task: Reliable Intelligence Identification on Vietnamese SNSs. With appropriate training techniques, our model is able to achieve 0.9392 ROC-AUC on public test set and the final version settles at top 2 ROC-AUC (0.9513) on private test set.

Related papers

• Fantastic Gains and Where to Find Them: On the Existence and Prospect of General Knowledge Transfer between Any Pretrained Model [74.62272538148245]
  We show that for arbitrary pairings of pretrained models, one model extracts significant data context unavailable in the other. We investigate if it is possible to transfer such "complementary" knowledge from one model to another without performance degradation.
  arXiv  Detail & Related papers  (2023-10-26T17:59:46Z)
• Architecture, Dataset and Model-Scale Agnostic Data-free Meta-Learning [117.48444197402858]
  We propose ePisode cUrriculum inveRsion (ECI) during data-free meta training and invErsion calibRation following inner loop (ICFIL) during meta testing. ECI adaptively increases the difficulty level of pseudo episodes according to the real-time feedback of the meta model. We formulate the optimization process of meta training with ECI as an adversarial form in an end-to-end manner.
  arXiv  Detail & Related papers  (2023-03-20T15:10:41Z)
• Decoupled Mixup for Generalized Visual Recognition [71.13734761715472]
  We propose a novel "Decoupled-Mixup" method to train CNN models for visual recognition. Our method decouples each image into discriminative and noise-prone regions, and then heterogeneously combines these regions to train CNN models. Experiment results show the high generalization performance of our method on testing data that are composed of unseen contexts.
  arXiv  Detail & Related papers  (2022-10-26T15:21:39Z)
• A Modified PINN Approach for Identifiable Compartmental Models in Epidemiology with Applications to COVID-19 [0.0]
  We present an approach toward analyzing accessible data on Covid-19's U.S. development using a variation of the "Physics Informed Neural Networks" Aspects of identifiability of the model parameters are also assessed, as well as methods of denoising available data using a wavelet transform.
  arXiv  Detail & Related papers  (2022-08-01T23:09:32Z)
• CONVIQT: Contrastive Video Quality Estimator [63.749184706461826]
  Perceptual video quality assessment (VQA) is an integral component of many streaming and video sharing platforms. Here we consider the problem of learning perceptually relevant video quality representations in a self-supervised manner. Our results indicate that compelling representations with perceptual bearing can be obtained using self-supervised learning.
  arXiv  Detail & Related papers  (2022-06-29T15:22:01Z)
• Robust Binary Models by Pruning Randomly-initialized Networks [57.03100916030444]
  We propose ways to obtain robust models against adversarial attacks from randomly-d binary networks. We learn the structure of the robust model by pruning a randomly-d binary network. Our method confirms the strong lottery ticket hypothesis in the presence of adversarial attacks.
  arXiv  Detail & Related papers  (2022-02-03T00:05:08Z)
• ReINTEL Challenge 2020: A Comparative Study of Hybrid Deep Neural Network for Reliable Intelligence Identification on Vietnamese SNSs [0.9697877942346906]
  The overwhelming abundance of data has created a misinformation crisis. We propose a multi-input model that can effectively leverage both tabular metadata and post content for the task. Applying state-of-the-art finetuning techniques for the pretrained component and training strategies for our complete model, we have achieved a 0.9462 ROC-score on the VLSP private test set.
  arXiv  Detail & Related papers  (2021-09-27T03:40:28Z)
• Siamese Neural Network with Joint Bayesian Model Structure for Speaker Verification [54.96267179988487]
  We propose a novel Siamese neural network (SiamNN) for speaker verification. Joint distribution of samples is first formulated based on a joint Bayesian (JB) based generative model. We further train the model parameters with the pair-wised samples as a binary discrimination task for speaker verification.
  arXiv  Detail & Related papers  (2021-04-07T09:17:29Z)
• ReINTEL Challenge 2020: A Multimodal Ensemble Model for Detecting Unreliable Information on Vietnamese SNS [0.0]
  We propose a novel multimodal ensemble model which combines two multimodal models to solve the task. Experimental results showed that our proposed multimodal ensemble model improved against single models in term of ROC AUC score.
  arXiv  Detail & Related papers  (2020-12-18T14:33:08Z)
• Leveraging Transfer Learning for Reliable Intelligence Identification on Vietnamese SNSs (ReINTEL) [0.8602553195689513]
  We exploit both of monolingual and multilingual pre-trained models. Our team achieved a score of 0.9378 at ROC-AUC metric in the private test set.
  arXiv  Detail & Related papers  (2020-12-10T15:43:50Z)
• Gradient-Based Adversarial Training on Transformer Networks for Detecting Check-Worthy Factual Claims [3.7543966923106438]
  We introduce the first adversarially-regularized, transformer-based claim spotter model. We obtain a 4.70 point F1-score improvement over current state-of-the-art models. We propose a method to apply adversarial training to transformer models.
  arXiv  Detail & Related papers  (2020-02-18T16:51:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.