AmaSQuAD: A Benchmark for Amharic Extractive Question Answering

• URL: http://arxiv.org/abs/2502.02047v1
• Date: Tue, 04 Feb 2025 06:27:39 GMT
• Title: AmaSQuAD: A Benchmark for Amharic Extractive Question Answering
• Authors: Nebiyou Daniel Hailemariam, Blessed Guda, Tsegazeab Tefferi,
• Abstract summary: This research presents a novel framework for translating extractive question-answering datasets into low-resource languages. The methodology addresses challenges related to misalignment between translated questions and answers. We fine-tune the XLM-R model on the AmaSQuAD synthetic dataset for Amharic Question-Answering.
• Score: 0.0
• License:
• Abstract: This research presents a novel framework for translating extractive question-answering datasets into low-resource languages, as demonstrated by the creation of the AmaSQuAD dataset, a translation of SQuAD 2.0 into Amharic. The methodology addresses challenges related to misalignment between translated questions and answers, as well as the presence of multiple answer instances in the translated context. For this purpose, we used cosine similarity utilizing embeddings from a fine-tuned BERT-based model for Amharic and Longest Common Subsequence (LCS). Additionally, we fine-tune the XLM-R model on the AmaSQuAD synthetic dataset for Amharic Question-Answering. The results show an improvement in baseline performance, with the fine-tuned model achieving an increase in the F1 score from 36.55% to 44.41% and 50.01% to 57.5% on the AmaSQuAD development dataset. Moreover, the model demonstrates improvement on the human-curated AmQA dataset, increasing the F1 score from 67.80% to 68.80% and the exact match score from 52.50% to 52.66%.The AmaSQuAD dataset is publicly available Datasets

Related papers

• Optimized Quran Passage Retrieval Using an Expanded QA Dataset and Fine-Tuned Language Models [0.0]
  The Qur'an QA 2023 shared task dataset had a limited number of questions with weak model retrieval. The original dataset, which contains 251 questions, was reviewed and expanded to 629 questions with question diversification and reformulation. Experiments fine-tuned transformer models, including AraBERT, RoBERTa, CAMeLBERT, AraELECTRA, and BERT.
  arXiv  Detail & Related papers  (2024-12-16T04:03:58Z)
• Multilingual Non-Factoid Question Answering with Answer Paragraph Selection [36.31301773167754]
  This work presents MuNfQuAD, a multilingual QuAD with non-factoid questions. The dataset comprises over 578K QA pairs across 38 languages.
  arXiv  Detail & Related papers  (2024-08-20T07:37:06Z)
• KET-QA: A Dataset for Knowledge Enhanced Table Question Answering [63.56707527868466]
  We propose to use a knowledge base (KB) as the external knowledge source for TableQA. Every question requires the integration of information from both the table and the sub-graph to be answered. We design a retriever-reasoner structured pipeline model to extract pertinent information from the vast knowledge sub-graph.
  arXiv  Detail & Related papers  (2024-05-13T18:26:32Z)
• UQA: Corpus for Urdu Question Answering [3.979019316355144]
  This paper introduces UQA, a novel dataset for question answering and text comprehension in Urdu. UQA is generated by translating the Stanford Question Answering dataset (SQuAD2.0), a large-scale English QA dataset. The paper describes the process of selecting and evaluating the best translation model among two candidates: Google Translator and Seamless M4T.
  arXiv  Detail & Related papers  (2024-05-02T16:44:31Z)
• Synthetic Dataset Creation and Fine-Tuning of Transformer Models for Question Answering in Serbian [0.0]
  We create the largest Serbian QA dataset of more than 87K samples, which we name SQuAD-sr. To acknowledge the script duality in Serbian, we generated both Cyrillic and Latin versions of the dataset. Best results were obtained by fine-tuning the BERTi'c model on our Latin SQuAD-sr dataset, achieving 73.91% Exact Match and 82.97% F1 score.
  arXiv  Detail & Related papers  (2024-04-12T17:27:54Z)
• GSQA: An End-to-End Model for Generative Spoken Question Answering [54.418723701886115]
  We introduce the first end-to-end Generative Spoken Question Answering (GSQA) model that empowers the system to engage in abstractive reasoning. Our model surpasses the previous extractive model by 3% on extractive QA datasets. Our GSQA model shows the potential to generalize to a broad spectrum of questions, thus further expanding the spoken question answering capabilities of abstractive QA.
  arXiv  Detail & Related papers  (2023-12-15T13:33:18Z)
• UNK-VQA: A Dataset and a Probe into the Abstention Ability of Multi-modal Large Models [55.22048505787125]
  This paper contributes a comprehensive dataset, called UNK-VQA. We first augment the existing data via deliberate perturbations on either the image or question. We then extensively evaluate the zero- and few-shot performance of several emerging multi-modal large models.
  arXiv  Detail & Related papers  (2023-10-17T02:38:09Z)
• PAXQA: Generating Cross-lingual Question Answering Examples at Training Scale [53.92008514395125]
  PAXQA (Projecting annotations for cross-lingual (x) QA) decomposes cross-lingual QA into two stages. We propose a novel use of lexically-constrained machine translation, in which constrained entities are extracted from the parallel bitexts. We show that models fine-tuned on these datasets outperform prior synthetic data generation models over several extractive QA datasets.
  arXiv  Detail & Related papers  (2023-04-24T15:46:26Z)
• TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance [71.76018597965378]
  We build a new large-scale Question Answering dataset containing both Tabular And Textual data, named TAT-QA. We propose a novel QA model termed TAGOP, which is capable of reasoning over both tables and text.
  arXiv  Detail & Related papers  (2021-05-17T06:12:06Z)
• When in Doubt, Ask: Generating Answerable and Unanswerable Questions, Unsupervised [0.0]
  Question Answering (QA) is key for making possible a robust communication between human and machine. Modern language models used for QA have surpassed the human-performance in several essential tasks. This paper studies augmenting human-made datasets with synthetic data as a way of surmounting this problem.
  arXiv  Detail & Related papers  (2020-10-04T15:56:44Z)
• Template-Based Question Generation from Retrieved Sentences for Improved Unsupervised Question Answering [98.48363619128108]
  We propose an unsupervised approach to training QA models with generated pseudo-training data. We show that generating questions for QA training by applying a simple template on a related, retrieved sentence rather than the original context sentence improves downstream QA performance.
  arXiv  Detail & Related papers  (2020-04-24T17:57:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.