Grammatical Error Correction for Low-Resource Languages: The Case of Zarma

• URL: http://arxiv.org/abs/2410.15539v1
• Date: Sun, 20 Oct 2024 23:51:36 GMT
• Title: Grammatical Error Correction for Low-Resource Languages: The Case of Zarma
• Authors: Mamadou K. Keita, Christopher Homan, Sofiane Abdoulaye Hamani, Adwoa Bremang, Marcos Zampieri, Habibatou Abdoulaye Alfari, Elysabhete Amadou Ibrahim, Dennis Owusu,
• Abstract summary: Grammatical error correction (GEC) is important for improving written materials for low-resource languages like Zarma. This study compares rule-based methods, machine translation (MT) models, and large language models (LLMs) for GEC in Zarma.
• Score: 8.057796934109938
• License:
• Abstract: Grammatical error correction (GEC) is important for improving written materials for low-resource languages like Zarma -- spoken by over 5 million people in West Africa. Yet it remains a challenging problem. This study compares rule-based methods, machine translation (MT) models, and large language models (LLMs) for GEC in Zarma. We evaluate each approach's effectiveness on our manually-built dataset of over 250,000 examples using synthetic and human-annotated data. Our experiments show that the MT-based approach using the M2M100 model outperforms others, achieving a detection rate of 95.82% and a suggestion accuracy of 78.90% in automatic evaluations, and scoring 3.0 out of 5.0 in logical/grammar error correction during MEs by native speakers. The rule-based method achieved perfect detection (100%) and high suggestion accuracy (96.27%) for spelling corrections but struggled with context-level errors. LLMs like MT5-small showed moderate performance with a detection rate of 90.62% and a suggestion accuracy of 57.15%. Our work highlights the potential of MT models to enhance GEC in low-resource languages, paving the way for more inclusive NLP tools.

Related papers

• End-to-End Transformer-based Automatic Speech Recognition for Northern Kurdish: A Pioneering Approach [1.3689715712707342]
  This paper introduces a study exploring the effectiveness of Whisper, a pre-trained ASR model, for Northern Kurdish (Kurmanji) an under-resourced language spoken in the Middle East. Using a Northern Kurdish fine-tuning speech corpus containing approximately 68 hours of validated transcribed data, our experiments demonstrate that the additional module fine-tuning strategy significantly improves ASR accuracy.
  arXiv  Detail & Related papers  (2024-10-19T11:46:30Z)
• Improving Multilingual ASR in the Wild Using Simple N-best Re-ranking [68.77659513993507]
  We present a simple and effective N-best re-ranking approach to improve multilingual ASR accuracy. Our results show spoken language identification accuracy improvements of 8.7% and 6.1%, respectively, and word error rates which are 3.3% and 2.0% lower on these benchmarks.
  arXiv  Detail & Related papers  (2024-09-27T03:31:32Z)
• Building Math Agents with Multi-Turn Iterative Preference Learning [56.71330214021884]
  This paper studies the complementary direct preference learning approach to further improve model performance. Existing direct preference learning algorithms are originally designed for the single-turn chat task. We introduce a multi-turn direct preference learning framework, tailored for this context.
  arXiv  Detail & Related papers  (2024-09-04T02:41:04Z)
• Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs [54.05511925104712]
  We propose a simple, effective, and data-efficient method called Step-DPO. Step-DPO treats individual reasoning steps as units for preference optimization rather than evaluating answers holistically. Our findings demonstrate that as few as 10K preference data pairs and fewer than 500 Step-DPO training steps can yield a nearly 3% gain in accuracy on MATH for models with over 70B parameters.
  arXiv  Detail & Related papers  (2024-06-26T17:43:06Z)
• Benchmarks Underestimate the Readiness of Multi-lingual Dialogue Agents [39.92509218078164]
  We show for the first time, that in-context learning is sufficient to tackle multilingual TOD. We test our approach on the multilingual TOD dataset X-RiSAWOZ, which has 12 domains in Chinese, English, French, Korean, Hindi, and code-mixed Hindi-English.
  arXiv  Detail & Related papers  (2024-05-28T05:33:13Z)
• AraSpell: A Deep Learning Approach for Arabic Spelling Correction [0.0]
  "AraSpell" is a framework for Arabic spelling correction using different seq2seq model architectures. It was trained on more than 6.9 Million Arabic sentences.
  arXiv  Detail & Related papers  (2024-05-11T10:36:28Z)
• Text Quality-Based Pruning for Efficient Training of Language Models [66.66259229732121]
  We propose a novel method for numerically evaluating text quality in large unlabelled NLP datasets. By proposing the text quality metric, the paper establishes a framework to identify and eliminate low-quality text instances. Experimental results over multiple models and datasets demonstrate the efficacy of this approach.
  arXiv  Detail & Related papers  (2024-04-26T18:01:25Z)
• Assessing the Efficacy of Grammar Error Correction: A Human Evaluation Approach in the Japanese Context [10.047123247001714]
  We evaluate the performance of the state-of-the-art sequence tagging grammar error detection and correction model (SeqTagger) With an automatic annotation toolkit, ERRANT, we first evaluated SeqTagger's performance on error correction with human expert correction as the benchmark. Results indicated a precision of 63.66% and a recall of 20.19% for error correction in the full dataset.
  arXiv  Detail & Related papers  (2024-02-28T06:43:43Z)
• The Devil is in the Errors: Leveraging Large Language Models for Fine-grained Machine Translation Evaluation [93.01964988474755]
  AutoMQM is a prompting technique which asks large language models to identify and categorize errors in translations. We study the impact of labeled data through in-context learning and finetuning. We then evaluate AutoMQM with PaLM-2 models, and we find that it improves performance compared to just prompting for scores.
  arXiv  Detail & Related papers  (2023-08-14T17:17:21Z)
• A transformer-based spelling error correction framework for Bangla and resource scarce Indic languages [2.5874041837241304]
  Spelling error correction is the task of identifying and rectifying misspelled words in texts. Earlier efforts on spelling error correction in Bangla and resource-scarce Indic languages focused on rule-based, statistical, and machine learning-based methods. We propose a novel detector-purificator-corrector, DPC based on denoising transformers by addressing previous issues.
  arXiv  Detail & Related papers  (2022-11-07T17:59:05Z)
• Few-shot Instruction Prompts for Pretrained Language Models to Detect Social Biases [55.45617404586874]
  We propose a few-shot instruction-based method for prompting pre-trained language models (LMs) We show that large LMs can detect different types of fine-grained biases with similar and sometimes superior accuracy to fine-tuned models.
  arXiv  Detail & Related papers  (2021-12-15T04:19:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.