AraFinNLP 2024: The First Arabic Financial NLP Shared Task
- URL: http://arxiv.org/abs/2407.09818v1
- Date: Sat, 13 Jul 2024 09:28:44 GMT
- Title: AraFinNLP 2024: The First Arabic Financial NLP Shared Task
- Authors: Sanad Malaysha, Mo El-Haj, Saad Ezzini, Mohammed Khalilia, Mustafa Jarrar, Sultan Almujaiwel, Ismail Berrada, Houda Bouamor,
- Abstract summary: A total of 45 unique teams registered for this shared task, with 11 of them actively participated in the test phase.
The winning team of Subtask 1 achieved F1 score of 0.8773, and the only team submitted in Subtask 2 achieved a 1.667 BLEU score.
- Score: 5.74170730791988
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The expanding financial markets of the Arab world require sophisticated Arabic NLP tools. To address this need within the banking domain, the Arabic Financial NLP (AraFinNLP) shared task proposes two subtasks: (i) Multi-dialect Intent Detection and (ii) Cross-dialect Translation and Intent Preservation. This shared task uses the updated ArBanking77 dataset, which includes about 39k parallel queries in MSA and four dialects. Each query is labeled with one or more of a common 77 intents in the banking domain. These resources aim to foster the development of robust financial Arabic NLP, particularly in the areas of machine translation and banking chat-bots. A total of 45 unique teams registered for this shared task, with 11 of them actively participated in the test phase. Specifically, 11 teams participated in Subtask 1, while only 1 team participated in Subtask 2. The winning team of Subtask 1 achieved F1 score of 0.8773, and the only team submitted in Subtask 2 achieved a 1.667 BLEU score.
Related papers
- WojoodNER 2024: The Second Arabic Named Entity Recognition Shared Task [13.55190646427114]
WojoodNER-2024 encompassed three subtasks: (i) Closed-Track Flat Fine-Grained NER, (ii) Closed-Track Nested Fine-Grained NER, and (iii) an Open-Track NER for the Israeli War on Gaza.
The winning teams achieved F-1 scores of 91% and 92% in the Flat Fine-Grained and Nested Fine-Grained Subtasks, respectively.
arXiv Detail & Related papers (2024-07-13T16:17:08Z) - NADI 2024: The Fifth Nuanced Arabic Dialect Identification Shared Task [28.40134178913119]
We describe the findings of the fifth Nuanced Arabic Dialect Identification Shared Task (NADI 2024)
NADI 2024 targeted both dialect identification cast as a multi-label task and identification of the Arabic level of dialectness.
Winning teams achieved 50.57 Ftextsubscript1 on Subtask1, 0.1403 RMSE for Subtask2, and 20.44 BLEU in Subtask3, respectively.
arXiv Detail & Related papers (2024-07-06T01:18:58Z) - SemEval-2024 Task 8: Multidomain, Multimodel and Multilingual Machine-Generated Text Detection [68.858931667807]
Subtask A is a binary classification task determining whether a text is written by a human or generated by a machine.
Subtask B is to detect the exact source of a text, discerning whether it is written by a human or generated by a specific LLM.
Subtask C aims to identify the changing point within a text, at which the authorship transitions from human to machine.
arXiv Detail & Related papers (2024-04-22T13:56:07Z) - SemEval 2024 -- Task 10: Emotion Discovery and Reasoning its Flip in
Conversation (EDiReF) [61.49972925493912]
SemEval-2024 Task 10 is a shared task centred on identifying emotions in code-mixed dialogues.
This task comprises three distinct subtasks - emotion recognition in conversation for code-mixed dialogues, emotion flip reasoning for code-mixed dialogues, and emotion flip reasoning for English dialogues.
A total of 84 participants engaged in this task, with the most adept systems attaining F1-scores of 0.70, 0.79, and 0.76 for the respective subtasks.
arXiv Detail & Related papers (2024-02-29T08:20:06Z) - ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabic [53.1913348687902]
We present ArabicMMLU, the first multi-task language understanding benchmark for Arabic language.
Our data comprises 40 tasks and 14,575 multiple-choice questions in Modern Standard Arabic (MSA)
Our evaluations of 35 models reveal substantial room for improvement, particularly among the best open-source models.
arXiv Detail & Related papers (2024-02-20T09:07:41Z) - ArAIEval Shared Task: Persuasion Techniques and Disinformation Detection
in Arabic Text [41.3267575540348]
We present an overview of the ArAIEval shared task, organized as part of the first Arabic 2023 conference co-located with EMNLP 2023.
ArAIEval offers two tasks over Arabic text: (i) persuasion technique detection, focusing on identifying persuasion techniques in tweets and news articles, and (ii) disinformation detection in binary and multiclass setups over tweets.
A total of 20 teams participated in the final evaluation phase, with 14 and 16 teams participating in Tasks 1 and 2, respectively.
arXiv Detail & Related papers (2023-11-06T15:21:19Z) - NADI 2023: The Fourth Nuanced Arabic Dialect Identification Shared Task [28.986040897360336]
We describe the findings of the fourth Nuanced Arabic Dialect Identification Shared Task (NADI 2023)
NADI 2023 targeted both dialect identification (Subtask 1) and dialect-to-MSA machine translation (Subtask 2 and Subtask 3).
We describe the methods employed by the participating teams and briefly offer an outlook for NADI.
arXiv Detail & Related papers (2023-10-24T18:41:24Z) - NADI 2022: The Third Nuanced Arabic Dialect Identification Shared Task [16.688997360734472]
Third Nuanced Arabic Dialect Identification Shared Task (NADI 2022)
NADI 2022 targeted both dialect identification (Subtask 1) and dialectal sentiment analysis (Subtask 2) at the country level.
Winning team achieved 27.06 F1 on Subtask 1 and F1=75.16 on Subtask 2.
arXiv Detail & Related papers (2022-10-18T04:31:05Z) - UrduFake@FIRE2021: Shared Track on Fake News Identification in Urdu [55.41644538483948]
This study reports the second shared task named as UrduFake@FIRE2021 on identifying fake news detection in Urdu language.
The proposed systems were based on various count-based features and used different classifiers as well as neural network architectures.
The gradient descent (SGD) algorithm outperformed other classifiers and achieved 0.679 F-score.
arXiv Detail & Related papers (2022-07-11T19:15:04Z) - Number Entity Recognition [65.80137628972312]
Numbers are essential components of text, like any other word tokens, from which natural language processing (NLP) models are built and deployed.
In this work, we attempt to tap this potential of state-of-the-art NLP models and transfer their ability to boost performance in related tasks.
Our proposed classification of numbers into entities helps NLP models perform well on several tasks, including a handcrafted Fill-In-The-Blank (FITB) task and on question answering using joint embeddings.
arXiv Detail & Related papers (2022-05-07T05:22:43Z) - NADI 2020: The First Nuanced Arabic Dialect Identification Shared Task [18.23153068720659]
We present the results and findings of the First Nuanced Arabic Dialect Identification Shared Task (NADI)
Data for the shared task covers a total of 100 provinces from 21 Arab countries and are collected from the Twitter domain.
NADI is the first shared task to target naturally-occurring fine-grained dialectal text at the sub-country level.
arXiv Detail & Related papers (2020-10-21T22:14:28Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.