Offensive Language Detection in Under-resourced Algerian Dialectal Arabic Language

• URL: http://arxiv.org/abs/2203.10024v1
• Date: Fri, 18 Mar 2022 15:42:21 GMT
• Title: Offensive Language Detection in Under-resourced Algerian Dialectal Arabic Language
• Authors: Oussama Boucherit and Kheireddine Abainia
• Abstract summary: We focus on the Algerian dialectal Arabic which is one of under-resourced languages. Due to the scarcity of works on the same language, we have built a new corpus regrouping more than 8.7k texts manually annotated as normal, abusive and offensive.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper addresses the problem of detecting the offensive and abusive content in Facebook comments, where we focus on the Algerian dialectal Arabic which is one of under-resourced languages. The latter has a variety of dialects mixed with different languages (i.e. Berber, French and English). In addition, we deal with texts written in both Arabic and Roman scripts (i.e. Arabizi). Due to the scarcity of works on the same language, we have built a new corpus regrouping more than 8.7k texts manually annotated as normal, abusive and offensive. We have conducted a series of experiments using the state-of-the-art classifiers of text categorisation, namely: BiLSTM, CNN, FastText, SVM and NB. The results showed acceptable performances, but the problem requires further investigation on linguistic features to increase the identification accuracy.

Related papers

• Bilingual Adaptation of Monolingual Foundation Models [48.859227944759986]
  We present an efficient method for adapting a monolingual Large Language Model (LLM) to another language. Our two-stage approach begins with expanding the vocabulary and training only the embeddings matrix. By continually pre-training on a mix of Arabic and English corpora, the model retains its proficiency in English while acquiring capabilities in Arabic.
  arXiv  Detail & Related papers  (2024-07-13T21:09:38Z)
• ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabic [51.922112625469836]
  We present datasetname, the first multi-task language understanding benchmark for the Arabic language. Our data comprises 40 tasks and 14,575 multiple-choice questions in Modern Standard Arabic (MSA) and is carefully constructed by collaborating with native speakers in the region. Our evaluations of 35 models reveal substantial room for improvement, particularly among the best open-source models.
  arXiv  Detail & Related papers  (2024-02-20T09:07:41Z)
• ALDi: Quantifying the Arabic Level of Dialectness of Text [17.37857915257019]
  We argue that Arabic speakers perceive a spectrum of dialectness, which we operationalize at the sentence level as the Arabic Level of Dialectness (ALDi) We provide a detailed analysis of AOC-ALDi and show that a model trained on it can effectively identify levels of dialectness on a range of other corpora.
  arXiv  Detail & Related papers  (2023-10-20T18:07:39Z)
• AceGPT, Localizing Large Language Models in Arabic [73.39989503874634]
  The paper proposes a comprehensive solution that includes pre-training with Arabic texts, Supervised Fine-Tuning (SFT) utilizing native Arabic instructions, and GPT-4 responses in Arabic. The goal is to cultivate culturally cognizant and value-aligned Arabic LLMs capable of accommodating the diverse, application-specific needs of Arabic-speaking communities.
  arXiv  Detail & Related papers  (2023-09-21T13:20:13Z)
• NusaWrites: Constructing High-Quality Corpora for Underrepresented and Extremely Low-Resource Languages [54.808217147579036]
  We conduct a case study on Indonesian local languages. We compare the effectiveness of online scraping, human translation, and paragraph writing by native speakers in constructing datasets. Our findings demonstrate that datasets generated through paragraph writing by native speakers exhibit superior quality in terms of lexical diversity and cultural content.
  arXiv  Detail & Related papers  (2023-09-19T14:42:33Z)
• N-Shot Benchmarking of Whisper on Diverse Arabic Speech Recognition [8.731646409966737]
  It is not clear how Whisper would fare under diverse conditions even on languages it was evaluated on such as Arabic. Our evaluation covers most publicly available Arabic speech data and is performed under n-shot finetuning. We also investigate the robustness of Whisper under completely novel conditions, such as in dialect-accented standard Arabic and in unseen dialects.
  arXiv  Detail & Related papers  (2023-06-05T14:09:25Z)
• Beyond Arabic: Software for Perso-Arabic Script Manipulation [67.31374614549237]
  We provide a set of finite-state transducer (FST) components and corresponding utilities for manipulating the writing systems of languages that use the Perso-Arabic script. The library also provides simple FST-based romanization and transliteration.
  arXiv  Detail & Related papers  (2023-01-26T20:37:03Z)
• Graphemic Normalization of the Perso-Arabic Script [47.429213930688086]
  This paper documents the challenges that Perso-Arabic presents beyond the best-documented languages. We focus on the situation in natural language processing (NLP), which is affected by multiple, often neglected, issues. We evaluate the effects of script normalization on eight languages from diverse language families in the Perso-Arabic script diaspora on machine translation and statistical language modeling tasks.
  arXiv  Detail & Related papers  (2022-10-21T21:59:44Z)
• Sentiment Analysis in Poems in Misurata Sub-dialect -- A Sentiment Detection in an Arabic Sub-dialect [0.0]
  This study focuses on detecting sentiment in poems written in Misurata Arabic sub-dialect spoken in Libya. The tools used to detect sentiment from the dataset are Sklearn as well as Mazajak sentiment tool 1.
  arXiv  Detail & Related papers  (2021-09-15T10:42:39Z)
• Automatic Arabic Dialect Identification Systems for Written Texts: A Survey [0.0]
  Arabic dialect identification is a specific task of natural language processing, aiming to automatically predict the Arabic dialect of a given text. In this paper, we present a comprehensive survey of Arabic dialect identification research in written texts. We review the traditional machine learning methods, deep learning architectures, and complex learning approaches to Arabic dialect identification.
  arXiv  Detail & Related papers  (2020-09-26T15:33:16Z)
• Can Multilingual Language Models Transfer to an Unseen Dialect? A Case Study on North African Arabizi [2.76240219662896]
  We study the ability of multilingual language models to process an unseen dialect. We take user generated North-African Arabic as our case study. We show in zero-shot and unsupervised adaptation scenarios that multilingual language models are able to transfer to such an unseen dialect.
  arXiv  Detail & Related papers  (2020-05-01T11:29:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.