NADI 2023: The Fourth Nuanced Arabic Dialect Identification Shared Task

• URL: http://arxiv.org/abs/2310.16117v1
• Date: Tue, 24 Oct 2023 18:41:24 GMT
• Title: NADI 2023: The Fourth Nuanced Arabic Dialect Identification Shared Task
• Authors: Muhammad Abdul-Mageed, AbdelRahim Elmadany, Chiyu Zhang, El Moatez Billah Nagoudi, Houda Bouamor, Nizar Habash
• Abstract summary: We describe the findings of the fourth Nuanced Arabic Dialect Identification Shared Task (NADI 2023) NADI 2023 targeted both dialect identification (Subtask 1) and dialect-to-MSA machine translation (Subtask 2 and Subtask 3). We describe the methods employed by the participating teams and briefly offer an outlook for NADI.
• Score: 28.986040897360336
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We describe the findings of the fourth Nuanced Arabic Dialect Identification Shared Task (NADI 2023). The objective of NADI is to help advance state-of-the-art Arabic NLP by creating opportunities for teams of researchers to collaboratively compete under standardized conditions. It does so with a focus on Arabic dialects, offering novel datasets and defining subtasks that allow for meaningful comparisons between different approaches. NADI 2023 targeted both dialect identification (Subtask 1) and dialect-to-MSA machine translation (Subtask 2 and Subtask 3). A total of 58 unique teams registered for the shared task, of whom 18 teams have participated (with 76 valid submissions during test phase). Among these, 16 teams participated in Subtask 1, 5 participated in Subtask 2, and 3 participated in Subtask 3. The winning teams achieved 87.27 F1 on Subtask 1, 14.76 Bleu in Subtask 2, and 21.10 Bleu in Subtask 3, respectively. Results show that all three subtasks remain challenging, thereby motivating future work in this area. We describe the methods employed by the participating teams and briefly offer an outlook for NADI.

Related papers

• WojoodNER 2024: The Second Arabic Named Entity Recognition Shared Task [13.55190646427114]
  WojoodNER-2024 encompassed three subtasks: (i) Closed-Track Flat Fine-Grained NER, (ii) Closed-Track Nested Fine-Grained NER, and (iii) an Open-Track NER for the Israeli War on Gaza. The winning teams achieved F-1 scores of 91% and 92% in the Flat Fine-Grained and Nested Fine-Grained Subtasks, respectively.
  arXiv  Detail & Related papers  (2024-07-13T16:17:08Z)
• NADI 2024: The Fifth Nuanced Arabic Dialect Identification Shared Task [28.40134178913119]
  We describe the findings of the fifth Nuanced Arabic Dialect Identification Shared Task (NADI 2024) NADI 2024 targeted both dialect identification cast as a multi-label task and identification of the Arabic level of dialectness. Winning teams achieved 50.57 Ftextsubscript1 on Subtask1, 0.1403 RMSE for Subtask2, and 20.44 BLEU in Subtask3, respectively.
  arXiv  Detail & Related papers  (2024-07-06T01:18:58Z)
• SemEval-2024 Task 8: Multidomain, Multimodel and Multilingual Machine-Generated Text Detection [68.858931667807]
  Subtask A is a binary classification task determining whether a text is written by a human or generated by a machine. Subtask B is to detect the exact source of a text, discerning whether it is written by a human or generated by a specific LLM. Subtask C aims to identify the changing point within a text, at which the authorship transitions from human to machine.
  arXiv  Detail & Related papers  (2024-04-22T13:56:07Z)
• SemEval 2024 -- Task 10: Emotion Discovery and Reasoning its Flip in Conversation (EDiReF) [61.49972925493912]
  SemEval-2024 Task 10 is a shared task centred on identifying emotions in code-mixed dialogues. This task comprises three distinct subtasks - emotion recognition in conversation for code-mixed dialogues, emotion flip reasoning for code-mixed dialogues, and emotion flip reasoning for English dialogues. A total of 84 participants engaged in this task, with the most adept systems attaining F1-scores of 0.70, 0.79, and 0.76 for the respective subtasks.
  arXiv  Detail & Related papers  (2024-02-29T08:20:06Z)
• ArAIEval Shared Task: Persuasion Techniques and Disinformation Detection in Arabic Text [41.3267575540348]
  We present an overview of the ArAIEval shared task, organized as part of the first Arabic 2023 conference co-located with EMNLP 2023. ArAIEval offers two tasks over Arabic text: (i) persuasion technique detection, focusing on identifying persuasion techniques in tweets and news articles, and (ii) disinformation detection in binary and multiclass setups over tweets. A total of 20 teams participated in the final evaluation phase, with 14 and 16 teams participating in Tasks 1 and 2, respectively.
  arXiv  Detail & Related papers  (2023-11-06T15:21:19Z)
• Findings of the WMT 2022 Shared Task on Translation Suggestion [63.457874930232926]
  We report the result of the first edition of the WMT shared task on Translation Suggestion. The task aims to provide alternatives for specific words or phrases given the entire documents generated by machine translation (MT) It consists two sub-tasks, namely, the naive translation suggestion and translation suggestion with hints.
  arXiv  Detail & Related papers  (2022-11-30T03:48:36Z)
• NADI 2022: The Third Nuanced Arabic Dialect Identification Shared Task [16.688997360734472]
  Third Nuanced Arabic Dialect Identification Shared Task (NADI 2022) NADI 2022 targeted both dialect identification (Subtask 1) and dialectal sentiment analysis (Subtask 2) at the country level. Winning team achieved 27.06 F1 on Subtask 1 and F1=75.16 on Subtask 2.
  arXiv  Detail & Related papers  (2022-10-18T04:31:05Z)
• SemEval-2021 Task 4: Reading Comprehension of Abstract Meaning [47.49596196559958]
  This paper introduces the SemEval-2021 shared task 4: Reading of Abstract Meaning (ReCAM) Given a passage and the corresponding question, a participating system is expected to choose the correct answer from five candidates of abstract concepts. Subtask 1 aims to evaluate how well a system can model concepts that cannot be directly perceived in the physical world. Subtask 2 focuses on models' ability in comprehending nonspecific concepts located high in a hypernym hierarchy. Subtask 3 aims to provide some insights into models' generalizability over the two types of abstractness.
  arXiv  Detail & Related papers  (2021-05-31T11:04:17Z)
• NADI 2021: The Second Nuanced Arabic Dialect Identification Shared Task [20.34810224205086]
  This Shared Task includes four subtasks: country-level Modern Standard Arabic (MSA) identification (Subtask 1.1), country-level dialect identification (Subtask 1.2), and province-level sub-dialect identification (Subtask 2.1) The dataset covers a total of 100 provinces from 21 Arab countries, collected from the Twitter domain. A total of 53 teams from 23 countries registered to participate in the tasks, thus reflecting the interest of the community in this area.
  arXiv  Detail & Related papers  (2021-03-04T04:59:37Z)
• NADI 2020: The First Nuanced Arabic Dialect Identification Shared Task [18.23153068720659]
  We present the results and findings of the First Nuanced Arabic Dialect Identification Shared Task (NADI) Data for the shared task covers a total of 100 provinces from 21 Arab countries and are collected from the Twitter domain. NADI is the first shared task to target naturally-occurring fine-grained dialectal text at the sub-country level.
  arXiv  Detail & Related papers  (2020-10-21T22:14:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.