Myanmar XNLI: Building a Dataset and Exploring Low-resource Approaches to Natural Language Inference with Myanmar

• URL: http://arxiv.org/abs/2504.09645v1
• Date: Sun, 13 Apr 2025 16:36:59 GMT
• Title: Myanmar XNLI: Building a Dataset and Exploring Low-resource Approaches to Natural Language Inference with Myanmar
• Authors: Aung Kyaw Htet, Mark Dras,
• Abstract summary: We extend the XNLI task for one additional low-resource language, Myanmar, as a proxy challenge for broader low-resource languages.<n>First, we build a dataset called Myanmar XNLI using community crowd-sourced methods.<n>Second, we carry out evaluations of recent multilingual language models on the myXNLI benchmark, as well as explore data-augmentation methods to improve model performance.
• Score: 2.8023035616913785
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite dramatic recent progress in NLP, it is still a major challenge to apply Large Language Models (LLM) to low-resource languages. This is made visible in benchmarks such as Cross-Lingual Natural Language Inference (XNLI), a key task that demonstrates cross-lingual capabilities of NLP systems across a set of 15 languages. In this paper, we extend the XNLI task for one additional low-resource language, Myanmar, as a proxy challenge for broader low-resource languages, and make three core contributions. First, we build a dataset called Myanmar XNLI (myXNLI) using community crowd-sourced methods, as an extension to the existing XNLI corpus. This involves a two-stage process of community-based construction followed by expert verification; through an analysis, we demonstrate and quantify the value of the expert verification stage in the context of community-based construction for low-resource languages. We make the myXNLI dataset available to the community for future research. Second, we carry out evaluations of recent multilingual language models on the myXNLI benchmark, as well as explore data-augmentation methods to improve model performance. Our data-augmentation methods improve model accuracy by up to 2 percentage points for Myanmar, while uplifting other languages at the same time. Third, we investigate how well these data-augmentation methods generalise to other low-resource languages in the XNLI dataset.

Related papers

• Cross-Lingual Transfer for Low-Resource Natural Language Processing [0.32634122554914]
  Cross-lingual transfer learning is a research area aimed at leveraging data and models from high-resource languages to improve NLP performance.<n>This thesis presents a new method to improve data-based transfer with T-Projection, a state-of-the-art annotation projection method.<n>For model-based transfer, we introduce a constrained decoding algorithm that enhances cross-lingual Sequence Labeling in zero-shot settings.<n>Finally, we develop Medical mT5, the first multilingual text-to-text medical model.
  arXiv  Detail & Related papers  (2025-02-04T21:17:46Z)
• MERaLiON-TextLLM: Cross-Lingual Understanding of Large Language Models in Chinese, Indonesian, Malay, and Singlish [17.36441080071885]
  This report presents MERaLiON-TextLLM, a series of open-source language models specifically tailored to improve understanding and generation in Chinese, Indonesian, Malay, and Singlish.<n>Our approach achieves performance improvements across benchmarks in these languages, exceeding the capabilities of the official Llama-3 models.
  arXiv  Detail & Related papers  (2024-12-21T05:50:48Z)
• High-quality Data-to-Text Generation for Severely Under-Resourced Languages with Out-of-the-box Large Language Models [5.632410663467911]
  We explore the extent to which pretrained large language models (LLMs) can bridge the performance gap for under-resourced languages. We find that LLMs easily set the state of the art for the under-resourced languages by substantial margins. For all our languages, human evaluation shows on-a-par performance with humans for our best systems, but BLEU scores collapse compared to English.
  arXiv  Detail & Related papers  (2024-02-19T16:29:40Z)
• Natural Language Processing for Dialects of a Language: A Survey [56.93337350526933]
  State-of-the-art natural language processing (NLP) models are trained on massive training corpora, and report a superlative performance on evaluation datasets. This survey delves into an important attribute of these datasets: the dialect of a language. Motivated by the performance degradation of NLP models for dialectal datasets and its implications for the equity of language technologies, we survey past research in NLP for dialects in terms of datasets, and approaches.
  arXiv  Detail & Related papers  (2024-01-11T03:04:38Z)
• YAYI 2: Multilingual Open-Source Large Language Models [53.92832054643197]
  We propose YAYI 2, including both base and chat models, with 30 billion parameters. YAYI 2 is pre-trained from scratch on a multilingual corpus which contains 2.65 trillion tokens filtered by our pre-training data processing pipeline. The base model is aligned with human values through supervised fine-tuning with millions of instructions and reinforcement learning from human feedback.
  arXiv  Detail & Related papers  (2023-12-22T17:34:47Z)
• PLUG: Leveraging Pivot Language in Cross-Lingual Instruction Tuning [46.153828074152436]
  We propose a pivot language guided generation approach to enhance instruction tuning in lower-resource languages. It trains the model to first process instructions in the pivot language, and then produce responses in the target language. Our approach demonstrates a significant improvement in the instruction-following abilities of LLMs by 29% on average.
  arXiv  Detail & Related papers  (2023-11-15T05:28:07Z)
• GlotLID: Language Identification for Low-Resource Languages [51.38634652914054]
  GlotLID-M is an LID model that satisfies the desiderata of wide coverage, reliability and efficiency. It identifies 1665 languages, a large increase in coverage compared to prior work.
  arXiv  Detail & Related papers  (2023-10-24T23:45:57Z)
• An Open Dataset and Model for Language Identification [84.15194457400253]
  We present a LID model which achieves a macro-average F1 score of 0.93 and a false positive rate of 0.033 across 201 languages. We make both the model and the dataset available to the research community.
  arXiv  Detail & Related papers  (2023-05-23T08:43:42Z)
• Multi-lingual Evaluation of Code Generation Models [82.7357812992118]
  We present new benchmarks on evaluation code generation models: MBXP and Multilingual HumanEval, and MathQA-X. These datasets cover over 10 programming languages. We are able to assess the performance of code generation models in a multi-lingual fashion.
  arXiv  Detail & Related papers  (2022-10-26T17:17:06Z)
• ZmBART: An Unsupervised Cross-lingual Transfer Framework for Language Generation [4.874780144224057]
  Cross-lingual transfer for natural language generation is relatively understudied. We consider four NLG tasks (text summarization, question generation, news headline generation, and distractor generation) and three syntactically diverse languages. We propose an unsupervised cross-lingual language generation framework (called ZmBART) that does not use any parallel or pseudo-parallel/back-translated data.
  arXiv  Detail & Related papers  (2021-06-03T05:08:01Z)
• UNKs Everywhere: Adapting Multilingual Language Models to New Scripts [103.79021395138423]
  Massively multilingual language models such as multilingual BERT (mBERT) and XLM-R offer state-of-the-art cross-lingual transfer performance on a range of NLP tasks. Due to their limited capacity and large differences in pretraining data, there is a profound performance gap between resource-rich and resource-poor target languages. We propose novel data-efficient methods that enable quick and effective adaptation of pretrained multilingual models to such low-resource languages and unseen scripts.
  arXiv  Detail & Related papers  (2020-12-31T11:37:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.