LLMs for Extremely Low-Resource Finno-Ugric Languages

• URL: http://arxiv.org/abs/2410.18902v1
• Date: Thu, 24 Oct 2024 16:48:12 GMT
• Title: LLMs for Extremely Low-Resource Finno-Ugric Languages
• Authors: Taido Purason, Hele-Andra Kuulmets, Mark Fishel,
• Abstract summary: This paper addresses the gap by focusing on Voro, Livonian, and Komi. We cover almost the entire cycle of LLM creation, from data collection to instruction tuning and evaluation. We intend for this work to promote linguistic diversity, ensuring that lesser-resourced languages can benefit from advancements in NLP.
• Score: 0.8192907805418583
• License:
• Abstract: The advancement of large language models (LLMs) has predominantly focused on high-resource languages, leaving low-resource languages, such as those in the Finno-Ugric family, significantly underrepresented. This paper addresses this gap by focusing on V\~oro, Livonian, and Komi. We cover almost the entire cycle of LLM creation, from data collection to instruction tuning and evaluation. Our contributions include developing multilingual base and instruction-tuned models; creating evaluation benchmarks, including the smugri-MT-bench multi-turn conversational benchmark; and conducting human evaluation. We intend for this work to promote linguistic diversity, ensuring that lesser-resourced languages can benefit from advancements in NLP.

Related papers

• Think Carefully and Check Again! Meta-Generation Unlocking LLMs for Low-Resource Cross-Lingual Summarization [108.6908427615402]
  Cross-lingual summarization ( CLS) aims to generate a summary for the source text in a different target language. Currently, instruction-tuned large language models (LLMs) excel at various English tasks. Recent studies have shown that LLMs' performance on CLS tasks remains unsatisfactory even with few-shot settings.
  arXiv  Detail & Related papers  (2024-10-26T00:39:44Z)
• Do Large Language Models Have an English Accent? Evaluating and Improving the Naturalness of Multilingual LLMs [13.558778781305998]
  Large Language Models (LLMs) are predominantly designed with English as the primary language. Even the few that are multilingual tend to exhibit strong English-centric biases. This paper introduces novel automatic corpus-level metrics to assess the lexical and syntactic naturalness of multilingual outputs.
  arXiv  Detail & Related papers  (2024-10-21T12:34:17Z)
• Generating bilingual example sentences with large language models as lexicography assistants [2.6550899846546527]
  We present a study of LLMs' performance in generating and rating example sentences for bilingual dictionaries across languages with varying resource levels. We evaluate the quality of LLM-generated examples against the GDEX (Good Dictionary EXample) criteria: typicality, informativeness, and intelligibility.
  arXiv  Detail & Related papers  (2024-10-04T06:45:48Z)
• Quantifying Multilingual Performance of Large Language Models Across Languages [48.40607157158246]
  Large Language Models (LLMs) perform better on high-resource languages like English, German, and French, while their capabilities in low-resource languages remain inadequate. We propose the Language Ranker, an intrinsic metric designed to benchmark and rank languages based on LLM performance using internal representations. Our analysis reveals that high-resource languages exhibit higher similarity scores with English, demonstrating superior performance, while low-resource languages show lower similarity scores.
  arXiv  Detail & Related papers  (2024-04-17T16:53:16Z)
• LLMs Are Few-Shot In-Context Low-Resource Language Learners [59.74451570590808]
  In-context learning (ICL) empowers large language models (LLMs) to perform diverse tasks in underrepresented languages. We extensively study ICL and its cross-lingual variation (X-ICL) on 25 low-resource and 7 relatively higher-resource languages. Our study concludes the significance of few-shot in-context information on enhancing the low-resource understanding quality of LLMs.
  arXiv  Detail & Related papers  (2024-03-25T07:55:29Z)
• OMGEval: An Open Multilingual Generative Evaluation Benchmark for Large Language Models [59.54423478596468]
  We introduce OMGEval, the first Open-source Multilingual Generative test set that can assess the capability of LLMs in different languages. For each language, OMGEval provides 804 open-ended questions, covering a wide range of important capabilities of LLMs. Specifically, the current version of OMGEval includes 5 languages (i.e., Zh, Ru, Fr, Es, Ar)
  arXiv  Detail & Related papers  (2024-02-21T04:42:41Z)
• High-quality Data-to-Text Generation for Severely Under-Resourced Languages with Out-of-the-box Large Language Models [5.632410663467911]
  We explore the extent to which pretrained large language models (LLMs) can bridge the performance gap for under-resourced languages. We find that LLMs easily set the state of the art for the under-resourced languages by substantial margins. For all our languages, human evaluation shows on-a-par performance with humans for our best systems, but BLEU scores collapse compared to English.
  arXiv  Detail & Related papers  (2024-02-19T16:29:40Z)
• Enhancing Multilingual Capabilities of Large Language Models through Self-Distillation from Resource-Rich Languages [60.162717568496355]
  Large language models (LLMs) have been pre-trained on multilingual corpora. Their performance still lags behind in most languages compared to a few resource-rich languages.
  arXiv  Detail & Related papers  (2024-02-19T15:07:32Z)
• PLUG: Leveraging Pivot Language in Cross-Lingual Instruction Tuning [46.153828074152436]
  We propose a pivot language guided generation approach to enhance instruction tuning in lower-resource languages. It trains the model to first process instructions in the pivot language, and then produce responses in the target language. Our approach demonstrates a significant improvement in the instruction-following abilities of LLMs by 29% on average.
  arXiv  Detail & Related papers  (2023-11-15T05:28:07Z)
• Democratizing LLMs for Low-Resource Languages by Leveraging their English Dominant Abilities with Linguistically-Diverse Prompts [75.33019401706188]
  Large language models (LLMs) are known to effectively perform tasks by simply observing few exemplars. We propose to assemble synthetic exemplars from a diverse set of high-resource languages to prompt the LLMs to translate from any language into English. Our unsupervised prompting method performs on par with supervised few-shot learning in LLMs of different sizes for translations between English and 13 Indic and 21 African low-resource languages.
  arXiv  Detail & Related papers  (2023-06-20T08:27:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.