Pre-trained Transformer-Based Approach for Arabic Question Answering : A Comparative Study

• URL: http://arxiv.org/abs/2111.05671v1
• Date: Wed, 10 Nov 2021 12:33:18 GMT
• Title: Pre-trained Transformer-Based Approach for Arabic Question Answering : A Comparative Study
• Authors: Kholoud Alsubhi, Amani Jamal, Areej Alhothali
• Abstract summary: We evaluate the state-of-the-art pre-trained transformers models for Arabic QA using four reading comprehension datasets. We fine-tuned and compared the performance of the AraBERTv2-base model, AraBERTv0.2-large model, and AraELECTRA model.
• Score: 0.5801044612920815
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Question answering(QA) is one of the most challenging yet widely investigated problems in Natural Language Processing (NLP). Question-answering (QA) systems try to produce answers for given questions. These answers can be generated from unstructured or structured text. Hence, QA is considered an important research area that can be used in evaluating text understanding systems. A large volume of QA studies was devoted to the English language, investigating the most advanced techniques and achieving state-of-the-art results. However, research efforts in the Arabic question-answering progress at a considerably slower pace due to the scarcity of research efforts in Arabic QA and the lack of large benchmark datasets. Recently many pre-trained language models provided high performance in many Arabic NLP problems. In this work, we evaluate the state-of-the-art pre-trained transformers models for Arabic QA using four reading comprehension datasets which are Arabic-SQuAD, ARCD, AQAD, and TyDiQA-GoldP datasets. We fine-tuned and compared the performance of the AraBERTv2-base model, AraBERTv0.2-large model, and AraELECTRA model. In the last, we provide an analysis to understand and interpret the low-performance results obtained by some models.

Related papers

• FairytaleQA Translated: Enabling Educational Question and Answer Generation in Less-Resourced Languages [0.0]
  This paper introduces machine-translated versions of FairytaleQA, a renowned QA dataset designed to assess and enhance narrative comprehension skills in young children. We employ fine-tuned, modest-scale models to establish benchmarks for both Question Generation (QG) and QA tasks within the translated datasets. We present a case study proposing a model for generating question-answer pairs, with an evaluation incorporating quality metrics such as question well-formedness, answerability, relevance, and children suitability.
  arXiv  Detail & Related papers  (2024-06-06T16:31:47Z)
• Can a Multichoice Dataset be Repurposed for Extractive Question Answering? [52.28197971066953]
  We repurposed the Belebele dataset (Bandarkar et al., 2023), which was designed for multiple-choice question answering (MCQA) We present annotation guidelines and a parallel EQA dataset for English and Modern Standard Arabic (MSA). Our aim is to enable others to adapt our approach for the 120+ other language variants in Belebele, many of which are deemed under-resourced.
  arXiv  Detail & Related papers  (2024-04-26T11:46:05Z)
• Building Efficient and Effective OpenQA Systems for Low-Resource Languages [17.64851283209797]
  We show that effective, low-cost OpenQA systems can be developed for low-resource contexts. Key ingredients are weak supervision using machine-translated labeled datasets and a relevant unstructured knowledge source. We present SQuAD-TR, a machine translation of SQuAD2.0, and we build our OpenQA system by adapting ColBERT-QA and retraining it over Turkish resources.
  arXiv  Detail & Related papers  (2024-01-07T22:11:36Z)
• AQUALLM: Audio Question Answering Data Generation Using Large Language Models [2.2232550112727267]
  We introduce a scalable AQA data generation pipeline, which relies on Large Language Models (LLMs) We present three extensive and high-quality benchmark datasets for AQA. Models trained on our datasets demonstrate enhanced generalizability when compared to models trained using human-annotated AQA data.
  arXiv  Detail & Related papers  (2023-12-28T20:01:27Z)
• Arabic Sentiment Analysis with Noisy Deep Explainable Model [48.22321420680046]
  This paper proposes an explainable sentiment classification framework for the Arabic language. The proposed framework can explain specific predictions by training a local surrogate explainable model. We carried out experiments on public benchmark Arabic SA datasets.
  arXiv  Detail & Related papers  (2023-09-24T19:26:53Z)
• An Empirical Comparison of LM-based Question and Answer Generation Methods [79.31199020420827]
  Question and answer generation (QAG) consists of generating a set of question-answer pairs given a context. In this paper, we establish baselines with three different QAG methodologies that leverage sequence-to-sequence language model (LM) fine-tuning. Experiments show that an end-to-end QAG model, which is computationally light at both training and inference times, is generally robust and outperforms other more convoluted approaches.
  arXiv  Detail & Related papers  (2023-05-26T14:59:53Z)
• PAXQA: Generating Cross-lingual Question Answering Examples at Training Scale [53.92008514395125]
  PAXQA (Projecting annotations for cross-lingual (x) QA) decomposes cross-lingual QA into two stages. We propose a novel use of lexically-constrained machine translation, in which constrained entities are extracted from the parallel bitexts. We show that models fine-tuned on these datasets outperform prior synthetic data generation models over several extractive QA datasets.
  arXiv  Detail & Related papers  (2023-04-24T15:46:26Z)
• QAmeleon: Multilingual QA with Only 5 Examples [71.80611036543633]
  We show how to leverage pre-trained language models under a few-shot learning setting. Our approach, QAmeleon, uses a PLM to automatically generate multilingual data upon which QA models are trained. Prompt tuning the PLM for data synthesis with only five examples per language delivers accuracy superior to translation-based baselines.
  arXiv  Detail & Related papers  (2022-11-15T16:14:39Z)
• Template-Based Question Generation from Retrieved Sentences for Improved Unsupervised Question Answering [98.48363619128108]
  We propose an unsupervised approach to training QA models with generated pseudo-training data. We show that generating questions for QA training by applying a simple template on a related, retrieved sentence rather than the original context sentence improves downstream QA performance.
  arXiv  Detail & Related papers  (2020-04-24T17:57:45Z)
• AraBERT: Transformer-based Model for Arabic Language Understanding [0.0]
  We pre-trained BERT specifically for the Arabic language in the pursuit of achieving the same success that BERT did for the English language. The results showed that the newly developed AraBERT achieved state-of-the-art performance on most tested Arabic NLP tasks.
  arXiv  Detail & Related papers  (2020-02-28T22:59:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.