AraFinNLP 2024: The First Arabic Financial NLP Shared Task

• URL: http://arxiv.org/abs/2407.09818v1
• Date: Sat, 13 Jul 2024 09:28:44 GMT
• Title: AraFinNLP 2024: The First Arabic Financial NLP Shared Task
• Authors: Sanad Malaysha, Mo El-Haj, Saad Ezzini, Mohammed Khalilia, Mustafa Jarrar, Sultan Almujaiwel, Ismail Berrada, Houda Bouamor,
• Abstract summary: A total of 45 unique teams registered for this shared task, with 11 of them actively participated in the test phase. The winning team of Subtask 1 achieved F1 score of 0.8773, and the only team submitted in Subtask 2 achieved a 1.667 BLEU score.
• Score: 5.74170730791988
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The expanding financial markets of the Arab world require sophisticated Arabic NLP tools. To address this need within the banking domain, the Arabic Financial NLP (AraFinNLP) shared task proposes two subtasks: (i) Multi-dialect Intent Detection and (ii) Cross-dialect Translation and Intent Preservation. This shared task uses the updated ArBanking77 dataset, which includes about 39k parallel queries in MSA and four dialects. Each query is labeled with one or more of a common 77 intents in the banking domain. These resources aim to foster the development of robust financial Arabic NLP, particularly in the areas of machine translation and banking chat-bots. A total of 45 unique teams registered for this shared task, with 11 of them actively participated in the test phase. Specifically, 11 teams participated in Subtask 1, while only 1 team participated in Subtask 2. The winning team of Subtask 1 achieved F1 score of 0.8773, and the only team submitted in Subtask 2 achieved a 1.667 BLEU score.

Related papers

• GenAI Content Detection Task 1: English and Multilingual Machine-Generated Text Detection: AI vs. Human [71.42669028683741]
  We present a shared task on binary machine generated text detection conducted as a part of the GenAI workshop at COLING 2025. The task consists of two subtasks: Monolingual (English) and Multilingual. We provide a comprehensive overview of the data, a summary of the results, detailed descriptions of the participating systems, and an in-depth analysis of submissions.
  arXiv  Detail & Related papers  (2025-01-19T11:11:55Z)
• Findings of the IWSLT 2024 Evaluation Campaign [102.7608597658451]
  The paper reports on the shared tasks organized by the 21st IWSLT Conference. The shared tasks address 7 scientific challenges in spoken language translation.
  arXiv  Detail & Related papers  (2024-11-07T19:11:55Z)
• Open-FinLLMs: Open Multimodal Large Language Models for Financial Applications [90.67346776473241]
  Large language models (LLMs) have advanced financial applications, yet they often lack sufficient financial knowledge and struggle with tasks involving multi-modal inputs like tables and time series data. We introduce textitOpen-FinLLMs, a series of Financial LLMs that embed comprehensive financial knowledge into text, tables, and time-series data. We also present FinLLaVA, a multimodal LLM trained with 1.43M image-text instructions to handle complex financial data types.
  arXiv  Detail & Related papers  (2024-08-20T16:15:28Z)
• WojoodNER 2024: The Second Arabic Named Entity Recognition Shared Task [13.55190646427114]
  WojoodNER-2024 encompassed three subtasks: (i) Closed-Track Flat Fine-Grained NER, (ii) Closed-Track Nested Fine-Grained NER, and (iii) an Open-Track NER for the Israeli War on Gaza. The winning teams achieved F-1 scores of 91% and 92% in the Flat Fine-Grained and Nested Fine-Grained Subtasks, respectively.
  arXiv  Detail & Related papers  (2024-07-13T16:17:08Z)
• NADI 2024: The Fifth Nuanced Arabic Dialect Identification Shared Task [28.40134178913119]
  We describe the findings of the fifth Nuanced Arabic Dialect Identification Shared Task (NADI 2024) NADI 2024 targeted both dialect identification cast as a multi-label task and identification of the Arabic level of dialectness. Winning teams achieved 50.57 Ftextsubscript1 on Subtask1, 0.1403 RMSE for Subtask2, and 20.44 BLEU in Subtask3, respectively.
  arXiv  Detail & Related papers  (2024-07-06T01:18:58Z)
• SemEval-2024 Task 8: Multidomain, Multimodel and Multilingual Machine-Generated Text Detection [68.858931667807]
  Subtask A is a binary classification task determining whether a text is written by a human or generated by a machine. Subtask B is to detect the exact source of a text, discerning whether it is written by a human or generated by a specific LLM. Subtask C aims to identify the changing point within a text, at which the authorship transitions from human to machine.
  arXiv  Detail & Related papers  (2024-04-22T13:56:07Z)
• ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabic [51.922112625469836]
  We present datasetname, the first multi-task language understanding benchmark for the Arabic language. Our data comprises 40 tasks and 14,575 multiple-choice questions in Modern Standard Arabic (MSA) and is carefully constructed by collaborating with native speakers in the region. Our evaluations of 35 models reveal substantial room for improvement, particularly among the best open-source models.
  arXiv  Detail & Related papers  (2024-02-20T09:07:41Z)
• NADI 2023: The Fourth Nuanced Arabic Dialect Identification Shared Task [28.986040897360336]
  We describe the findings of the fourth Nuanced Arabic Dialect Identification Shared Task (NADI 2023) NADI 2023 targeted both dialect identification (Subtask 1) and dialect-to-MSA machine translation (Subtask 2 and Subtask 3). We describe the methods employed by the participating teams and briefly offer an outlook for NADI.
  arXiv  Detail & Related papers  (2023-10-24T18:41:24Z)
• NADI 2022: The Third Nuanced Arabic Dialect Identification Shared Task [16.688997360734472]
  Third Nuanced Arabic Dialect Identification Shared Task (NADI 2022) NADI 2022 targeted both dialect identification (Subtask 1) and dialectal sentiment analysis (Subtask 2) at the country level. Winning team achieved 27.06 F1 on Subtask 1 and F1=75.16 on Subtask 2.
  arXiv  Detail & Related papers  (2022-10-18T04:31:05Z)
• UrduFake@FIRE2021: Shared Track on Fake News Identification in Urdu [55.41644538483948]
  This study reports the second shared task named as UrduFake@FIRE2021 on identifying fake news detection in Urdu language. The proposed systems were based on various count-based features and used different classifiers as well as neural network architectures. The gradient descent (SGD) algorithm outperformed other classifiers and achieved 0.679 F-score.
  arXiv  Detail & Related papers  (2022-07-11T19:15:04Z)
• Number Entity Recognition [65.80137628972312]
  Numbers are essential components of text, like any other word tokens, from which natural language processing (NLP) models are built and deployed. In this work, we attempt to tap this potential of state-of-the-art NLP models and transfer their ability to boost performance in related tasks. Our proposed classification of numbers into entities helps NLP models perform well on several tasks, including a handcrafted Fill-In-The-Blank (FITB) task and on question answering using joint embeddings.
  arXiv  Detail & Related papers  (2022-05-07T05:22:43Z)
• NADI 2020: The First Nuanced Arabic Dialect Identification Shared Task [18.23153068720659]
  We present the results and findings of the First Nuanced Arabic Dialect Identification Shared Task (NADI) Data for the shared task covers a total of 100 provinces from 21 Arab countries and are collected from the Twitter domain. NADI is the first shared task to target naturally-occurring fine-grained dialectal text at the sub-country level.
  arXiv  Detail & Related papers  (2020-10-21T22:14:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.