Lifelong Learning Natural Language Processing Approach for Multilingual
Data Classification
- URL: http://arxiv.org/abs/2206.11867v1
- Date: Wed, 25 May 2022 10:34:04 GMT
- Title: Lifelong Learning Natural Language Processing Approach for Multilingual
Data Classification
- Authors: J\k{e}drzej Kozal, Micha{\l} Le\'s, Pawe{\l} Zyblewski, Pawe{\l}
Ksieniewicz, Micha{\l} Wo\'zniak
- Abstract summary: We propose a lifelong learning-inspired approach, which allows for fake news detection in multiple languages.
The ability of models to generalize the knowledge acquired between the analyzed languages was also observed.
- Score: 1.3999481573773074
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The abundance of information in digital media, which in today's world is the
main source of knowledge about current events for the masses, makes it possible
to spread disinformation on a larger scale than ever before. Consequently,
there is a need to develop novel fake news detection approaches capable of
adapting to changing factual contexts and generalizing previously or
concurrently acquired knowledge. To deal with this problem, we propose a
lifelong learning-inspired approach, which allows for fake news detection in
multiple languages and the mutual transfer of knowledge acquired in each of
them. Both classical feature extractors, such as Term frequency-inverse
document frequency or Latent Dirichlet Allocation, and integrated deep NLP
(Natural Language Processing) BERT (Bidirectional Encoder Representations from
Transformers) models paired with MLP (Multilayer Perceptron) classifier, were
employed. The results of experiments conducted on two datasets dedicated to the
fake news classification task (in English and Spanish, respectively), supported
by statistical analysis, confirmed that utilization of additional languages
could improve performance for traditional methods. Also, in some cases
supplementing the deep learning method with classical ones can positively
impact obtained results. The ability of models to generalize the knowledge
acquired between the analyzed languages was also observed.
Related papers
- Ethio-Fake: Cutting-Edge Approaches to Combat Fake News in Under-Resourced Languages Using Explainable AI [44.21078435758592]
Misinformation can spread quickly due to the ease of creating and disseminating content.
Traditional approaches to fake news detection often rely solely on content-based features.
We propose a comprehensive approach that integrates social context-based features with news content features.
arXiv Detail & Related papers (2024-10-03T15:49:35Z) - Commonsense Knowledge Transfer for Pre-trained Language Models [83.01121484432801]
We introduce commonsense knowledge transfer, a framework to transfer the commonsense knowledge stored in a neural commonsense knowledge model to a general-purpose pre-trained language model.
It first exploits general texts to form queries for extracting commonsense knowledge from the neural commonsense knowledge model.
It then refines the language model with two self-supervised objectives: commonsense mask infilling and commonsense relation prediction.
arXiv Detail & Related papers (2023-06-04T15:44:51Z) - Beyond Contrastive Learning: A Variational Generative Model for
Multilingual Retrieval [109.62363167257664]
We propose a generative model for learning multilingual text embeddings.
Our model operates on parallel data in $N$ languages.
We evaluate this method on a suite of tasks including semantic similarity, bitext mining, and cross-lingual question retrieval.
arXiv Detail & Related papers (2022-12-21T02:41:40Z) - Multiverse: Multilingual Evidence for Fake News Detection [71.51905606492376]
Multiverse is a new feature based on multilingual evidence that can be used for fake news detection.
The hypothesis of the usage of cross-lingual evidence as a feature for fake news detection is confirmed.
arXiv Detail & Related papers (2022-11-25T18:24:17Z) - On the cross-lingual transferability of multilingual prototypical models
across NLU tasks [2.44288434255221]
Supervised deep learning-based approaches have been applied to task-oriented dialog and have proven to be effective for limited domain and language applications.
In practice, these approaches suffer from the drawbacks of domain-driven design and under-resourced languages.
This article proposes to investigate the cross-lingual transferability of using synergistically few-shot learning with prototypical neural networks and multilingual Transformers-based models.
arXiv Detail & Related papers (2022-07-19T09:55:04Z) - Cross-lingual Lifelong Learning [53.06904052325966]
We present a principled Cross-lingual Continual Learning (CCL) evaluation paradigm.
We provide insights into what makes multilingual sequential learning particularly challenging.
The implications of this analysis include a recipe for how to measure and balance different cross-lingual continual learning desiderata.
arXiv Detail & Related papers (2022-05-23T09:25:43Z) - Detecting Text Formality: A Study of Text Classification Approaches [78.11745751651708]
This work proposes the first to our knowledge systematic study of formality detection methods based on statistical, neural-based, and Transformer-based machine learning methods.
We conducted three types of experiments -- monolingual, multilingual, and cross-lingual.
The study shows the overcome of Char BiLSTM model over Transformer-based ones for the monolingual and multilingual formality classification task.
arXiv Detail & Related papers (2022-04-19T16:23:07Z) - Knowledge Based Multilingual Language Model [44.70205282863062]
We present a novel framework to pretrain knowledge based multilingual language models (KMLMs)
We generate a large amount of code-switched synthetic sentences and reasoning-based multilingual training data using the Wikidata knowledge graphs.
Based on the intra- and inter-sentence structures of the generated data, we design pretraining tasks to facilitate knowledge learning.
arXiv Detail & Related papers (2021-11-22T02:56:04Z) - Meta-Transfer Learning for Code-Switched Speech Recognition [72.84247387728999]
We propose a new learning method, meta-transfer learning, to transfer learn on a code-switched speech recognition system in a low-resource setting.
Our model learns to recognize individual languages, and transfer them so as to better recognize mixed-language speech by conditioning the optimization on the code-switching data.
arXiv Detail & Related papers (2020-04-29T14:27:19Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.