Automatic Arabic Dialect Identification Systems for Written Texts: A Survey

• URL: http://arxiv.org/abs/2009.12622v1
• Date: Sat, 26 Sep 2020 15:33:16 GMT
• Title: Automatic Arabic Dialect Identification Systems for Written Texts: A Survey
• Authors: Maha J. Althobaiti
• Abstract summary: Arabic dialect identification is a specific task of natural language processing, aiming to automatically predict the Arabic dialect of a given text. In this paper, we present a comprehensive survey of Arabic dialect identification research in written texts. We review the traditional machine learning methods, deep learning architectures, and complex learning approaches to Arabic dialect identification.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Arabic dialect identification is a specific task of natural language processing, aiming to automatically predict the Arabic dialect of a given text. Arabic dialect identification is the first step in various natural language processing applications such as machine translation, multilingual text-to-speech synthesis, and cross-language text generation. Therefore, in the last decade, interest has increased in addressing the problem of Arabic dialect identification. In this paper, we present a comprehensive survey of Arabic dialect identification research in written texts. We first define the problem and its challenges. Then, the survey extensively discusses in a critical manner many aspects related to Arabic dialect identification task. So, we review the traditional machine learning methods, deep learning architectures, and complex learning approaches to Arabic dialect identification. We also detail the features and techniques for feature representations used to train the proposed systems. Moreover, we illustrate the taxonomy of Arabic dialects studied in the literature, the various levels of text processing at which Arabic dialect identification are conducted (e.g., token, sentence, and document level), as well as the available annotated resources, including evaluation benchmark corpora. Open challenges and issues are discussed at the end of the survey.

Related papers

• Exploiting Dialect Identification in Automatic Dialectal Text Normalization [9.320305816520422]
  We aim to normalize Dialectal Arabic into the Conventional Orthography for Dialectal Arabic (CODA) We benchmark newly developed sequence-to-sequence models on the task of CODAfication. We show that using dialect identification information improves the performance across all dialects.
  arXiv  Detail & Related papers  (2024-07-03T11:30:03Z)
• Can a Multichoice Dataset be Repurposed for Extractive Question Answering? [52.28197971066953]
  We repurposed the Belebele dataset (Bandarkar et al., 2023), which was designed for multiple-choice question answering (MCQA) We present annotation guidelines and a parallel EQA dataset for English and Modern Standard Arabic (MSA). Our aim is to enable others to adapt our approach for the 120+ other language variants in Belebele, many of which are deemed under-resourced.
  arXiv  Detail & Related papers  (2024-04-26T11:46:05Z)
• ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabic [51.922112625469836]
  We present datasetname, the first multi-task language understanding benchmark for the Arabic language. Our data comprises 40 tasks and 14,575 multiple-choice questions in Modern Standard Arabic (MSA) and is carefully constructed by collaborating with native speakers in the region. Our evaluations of 35 models reveal substantial room for improvement, particularly among the best open-source models.
  arXiv  Detail & Related papers  (2024-02-20T09:07:41Z)
• Natural Language Processing for Dialects of a Language: A Survey [56.93337350526933]
  State-of-the-art natural language processing (NLP) models are trained on massive training corpora, and report a superlative performance on evaluation datasets. This survey delves into an important attribute of these datasets: the dialect of a language. Motivated by the performance degradation of NLP models for dialectic datasets and its implications for the equity of language technologies, we survey past research in NLP for dialects in terms of datasets, and approaches.
  arXiv  Detail & Related papers  (2024-01-11T03:04:38Z)
• ALDi: Quantifying the Arabic Level of Dialectness of Text [17.37857915257019]
  We argue that Arabic speakers perceive a spectrum of dialectness, which we operationalize at the sentence level as the Arabic Level of Dialectness (ALDi) We provide a detailed analysis of AOC-ALDi and show that a model trained on it can effectively identify levels of dialectness on a range of other corpora.
  arXiv  Detail & Related papers  (2023-10-20T18:07:39Z)
• Multimodal Modeling For Spoken Language Identification [57.94119986116947]
  Spoken language identification refers to the task of automatically predicting the spoken language in a given utterance. We propose MuSeLI, a Multimodal Spoken Language Identification method, which delves into the use of various metadata sources to enhance language identification.
  arXiv  Detail & Related papers  (2023-09-19T12:21:39Z)
• Beyond Arabic: Software for Perso-Arabic Script Manipulation [67.31374614549237]
  We provide a set of finite-state transducer (FST) components and corresponding utilities for manipulating the writing systems of languages that use the Perso-Arabic script. The library also provides simple FST-based romanization and transliteration.
  arXiv  Detail & Related papers  (2023-01-26T20:37:03Z)
• Automated Audio Captioning: an Overview of Recent Progress and New Challenges [56.98522404673527]
  Automated audio captioning is a cross-modal translation task that aims to generate natural language descriptions for given audio clips. We present a comprehensive review of the published contributions in automated audio captioning, from a variety of existing approaches to evaluation metrics and datasets.
  arXiv  Detail & Related papers  (2022-05-12T08:36:35Z)
• Offensive Language Detection in Under-resourced Algerian Dialectal Arabic Language [0.0]
  We focus on the Algerian dialectal Arabic which is one of under-resourced languages. Due to the scarcity of works on the same language, we have built a new corpus regrouping more than 8.7k texts manually annotated as normal, abusive and offensive.
  arXiv  Detail & Related papers  (2022-03-18T15:42:21Z)
• Sentiment Analysis in Poems in Misurata Sub-dialect -- A Sentiment Detection in an Arabic Sub-dialect [0.0]
  This study focuses on detecting sentiment in poems written in Misurata Arabic sub-dialect spoken in Libya. The tools used to detect sentiment from the dataset are Sklearn as well as Mazajak sentiment tool 1.
  arXiv  Detail & Related papers  (2021-09-15T10:42:39Z)
• TArC: Incrementally and Semi-Automatically Collecting a Tunisian Arabish Corpus [3.8580784887142774]
  This article describes the constitution process of the first morpho-syntactically annotated Tunisian Arabish Corpus (TArC) Arabish, also known as Arabizi, is a spontaneous coding of Arabic dialects in Latin characters and arithmographs (numbers used as letters)
  arXiv  Detail & Related papers  (2020-03-20T22:29:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.