A Benchmark Arabic Dataset for Commonsense Explanation
- URL: http://arxiv.org/abs/2012.10251v1
- Date: Fri, 18 Dec 2020 14:07:10 GMT
- Title: A Benchmark Arabic Dataset for Commonsense Explanation
- Authors: Saja AL-Tawalbeh, Mohammad AL-Smadi
- Abstract summary: This paper presents a benchmark Arabic dataset for commonsense explanation.
The dataset consists of Arabic sentences that does not make sense along with three choices to select among them the one that explains why the sentence is false.
- Score: 0.6091702876917281
- License: http://creativecommons.org/publicdomain/zero/1.0/
- Abstract: Language comprehension and commonsense knowledge validation by machines are
challenging tasks that are still under researched and evaluated for Arabic
text. In this paper, we present a benchmark Arabic dataset for commonsense
explanation. The dataset consists of Arabic sentences that does not make sense
along with three choices to select among them the one that explains why the
sentence is false. Furthermore, this paper presents baseline results to assist
and encourage the future evaluation of research in this field. The dataset is
distributed under the Creative Commons CC-BY-SA 4.0 license and can be found on
GitHub
Related papers
- LexMatcher: Dictionary-centric Data Collection for LLM-based Machine Translation [67.24113079928668]
We present LexMatcher, a method for data curation driven by the coverage of senses found in bilingual dictionaries.
Our approach outperforms the established baselines on the WMT2022 test sets.
arXiv Detail & Related papers (2024-06-03T15:30:36Z) - ArabicaQA: A Comprehensive Dataset for Arabic Question Answering [13.65056111661002]
We introduce ArabicaQA, the first large-scale dataset for machine reading comprehension and open-domain question answering in Arabic.
We also present AraDPR, the first dense passage retrieval model trained on the Arabic Wikipedia corpus.
arXiv Detail & Related papers (2024-03-26T16:37:54Z) - ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabic [53.1913348687902]
We present ArabicMMLU, the first multi-task language understanding benchmark for Arabic language.
Our data comprises 40 tasks and 14,575 multiple-choice questions in Modern Standard Arabic (MSA)
Our evaluations of 35 models reveal substantial room for improvement, particularly among the best open-source models.
arXiv Detail & Related papers (2024-02-20T09:07:41Z) - AraSpider: Democratizing Arabic-to-SQL [1.082634245716027]
This study presents AraNLP, the first Arabic version of the Spider dataset, aimed at improving natural language processing (Spider) in the Arabic-speaking community.
arXiv Detail & Related papers (2024-02-12T07:11:13Z) - ARCOQ: Arabic Closest Opposite Questions Dataset [0.0]
This paper presents a dataset for closest opposite questions in Arabic language.
The structure is similar to that of the Graduate Record Examination (GRE) closest opposite questions dataset for the English language.
The paper provides a benchmark for the performance of different Arabic word embedding models on the introduced dataset.
arXiv Detail & Related papers (2023-10-22T18:41:26Z) - AceGPT, Localizing Large Language Models in Arabic [73.39989503874634]
The paper proposes a comprehensive solution that includes pre-training with Arabic texts, Supervised Fine-Tuning (SFT) utilizing native Arabic instructions, and GPT-4 responses in Arabic.
The goal is to cultivate culturally cognizant and value-aligned Arabic LLMs capable of accommodating the diverse, application-specific needs of Arabic-speaking communities.
arXiv Detail & Related papers (2023-09-21T13:20:13Z) - Comprehensive Benchmark Datasets for Amharic Scene Text Detection and
Recognition [56.048783994698425]
Ethiopic/Amharic script is one of the oldest African writing systems, which serves at least 23 languages in East Africa.
The Amharic writing system, Abugida, has 282 syllables, 15 punctuation marks, and 20 numerals.
We presented the first comprehensive public datasets named HUST-ART, HUST-AST, ABE, and Tana for Amharic script detection and recognition in the natural scene.
arXiv Detail & Related papers (2022-03-23T03:19:35Z) - New Arabic Medical Dataset for Diseases Classification [55.41644538483948]
We introduce a new Arab medical dataset, which includes two thousand medical documents collected from several Arabic medical websites.
The dataset was built for the task of classifying texts and includes 10 classes (Blood, Bone, Cardiovascular, Ear, Endocrine, Eye, Gastrointestinal, Immune, Liver and Nephrological)
Experiments on the dataset were performed by fine-tuning three pre-trained models: BERT from Google, Arabert that based on BERT with large Arabic corpus, and AraBioNER that based on Arabert with Arabic medical corpus.
arXiv Detail & Related papers (2021-06-29T10:42:53Z) - Is this sentence valid? An Arabic Dataset for Commonsense Validation [0.456877715768796]
This dataset is considered as the first in the field of Arabic text commonsense validation.
The dataset is distributed under the Creative Commons BY-SA 4.0 license and can be found on GitHub.
arXiv Detail & Related papers (2020-08-25T08:15:55Z) - AraDIC: Arabic Document Classification using Image-Based Character
Embeddings and Class-Balanced Loss [7.734726150561088]
We propose a novel end-to-end Arabic document classification framework, Arabic document image-based classifier (AraDIC)
AraDIC consists of an image-based character encoder and a classifier. They are trained in an end-to-end fashion using the class balanced loss to deal with the long-tailed data distribution problem.
To the best of our knowledge, this is the first image-based character embedding framework addressing the problem of Arabic text classification.
arXiv Detail & Related papers (2020-06-20T14:25:06Z) - ORB: An Open Reading Benchmark for Comprehensive Evaluation of Machine
Reading Comprehension [53.037401638264235]
We present an evaluation server, ORB, that reports performance on seven diverse reading comprehension datasets.
The evaluation server places no restrictions on how models are trained, so it is a suitable test bed for exploring training paradigms and representation learning.
arXiv Detail & Related papers (2019-12-29T07:27:23Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.