KyrgyzNLP: Challenges, Progress, and Future

• URL: http://arxiv.org/abs/2411.05503v2
• Date: Sat, 16 Nov 2024 03:53:07 GMT
• Title: KyrgyzNLP: Challenges, Progress, and Future
• Authors: Anton Alekseev, Timur Turatali,
• Abstract summary: Large language models (LLMs) have excelled in numerous benchmarks, advancing AI applications in both linguistic and non-linguistic tasks. This has primarily benefited well-resourced languages, leaving less-resourced ones (LRLs) at a disadvantage. In this paper, we highlight the current state of the NLP field in the specific LRL: kyrgyz tili.
• Score: 1.1920184024241331
• License:
• Abstract: Large language models (LLMs) have excelled in numerous benchmarks, advancing AI applications in both linguistic and non-linguistic tasks. However, this has primarily benefited well-resourced languages, leaving less-resourced ones (LRLs) at a disadvantage. In this paper, we highlight the current state of the NLP field in the specific LRL: kyrgyz tili. Human evaluation, including annotated datasets created by native speakers, remains an irreplaceable component of reliable NLP performance, especially for LRLs where automatic evaluations can fall short. In recent assessments of the resources for Turkic languages, Kyrgyz is labeled with the status 'Scraping By', a severely under-resourced language spoken by millions. This is concerning given the growing importance of the language, not only in Kyrgyzstan but also among diaspora communities where it holds no official status. We review prior efforts in the field, noting that many of the publicly available resources have only recently been developed, with few exceptions beyond dictionaries (the processed data used for the analysis is presented at https://kyrgyznlp.github.io/). While recent papers have made some headway, much more remains to be done. Despite interest and support from both business and government sectors in the Kyrgyz Republic, the situation for Kyrgyz language resources remains challenging. We stress the importance of community-driven efforts to build these resources, ensuring the future advancement sustainability. We then share our view of the most pressing challenges in Kyrgyz NLP. Finally, we propose a roadmap for future development in terms of research topics and language resources.

Related papers

• State of NLP in Kenya: A Survey [0.25454395163615406]
  Kenya, known for its linguistic diversity, faces unique challenges and promising opportunities in advancing Natural Language Processing. This survey provides a detailed assessment of the current state of NLP in Kenya. The paper uncovers significant gaps by critically evaluating the available datasets and existing NLP models.
  arXiv  Detail & Related papers  (2024-10-13T18:08:24Z)
• Recent Advancements and Challenges of Turkic Central Asian Language Processing [4.189204855014775]
  Research in NLP for Central Asian Turkic languages faces typical low-resource language challenges. Recent advancements have included the collection of language-specific datasets and the development of models for downstream tasks.
  arXiv  Detail & Related papers  (2024-07-06T08:58:26Z)
• Can a Multichoice Dataset be Repurposed for Extractive Question Answering? [52.28197971066953]
  We repurposed the Belebele dataset (Bandarkar et al., 2023), which was designed for multiple-choice question answering (MCQA) We present annotation guidelines and a parallel EQA dataset for English and Modern Standard Arabic (MSA). Our aim is to enable others to adapt our approach for the 120+ other language variants in Belebele, many of which are deemed under-resourced.
  arXiv  Detail & Related papers  (2024-04-26T11:46:05Z)
• LLMs Are Few-Shot In-Context Low-Resource Language Learners [59.74451570590808]
  In-context learning (ICL) empowers large language models (LLMs) to perform diverse tasks in underrepresented languages. We extensively study ICL and its cross-lingual variation (X-ICL) on 25 low-resource and 7 relatively higher-resource languages. Our study concludes the significance of few-shot in-context information on enhancing the low-resource understanding quality of LLMs.
  arXiv  Detail & Related papers  (2024-03-25T07:55:29Z)
• NusaWrites: Constructing High-Quality Corpora for Underrepresented and Extremely Low-Resource Languages [54.808217147579036]
  We conduct a case study on Indonesian local languages. We compare the effectiveness of online scraping, human translation, and paragraph writing by native speakers in constructing datasets. Our findings demonstrate that datasets generated through paragraph writing by native speakers exhibit superior quality in terms of lexical diversity and cultural content.
  arXiv  Detail & Related papers  (2023-09-19T14:42:33Z)
• A Survey of Corpora for Germanic Low-Resource Languages and Dialects [18.210880703295253]
  This work focuses on low-resource languages and in particular non-standardized low-resource languages. We make our overview of over 80 corpora publicly available to facilitate research.
  arXiv  Detail & Related papers  (2023-04-19T16:45:16Z)
• NusaX: Multilingual Parallel Sentiment Dataset for 10 Indonesian Local Languages [100.59889279607432]
  We focus on developing resources for languages in Indonesia. Most languages in Indonesia are categorized as endangered and some are even extinct. We develop the first-ever parallel resource for 10 low-resource languages in Indonesia.
  arXiv  Detail & Related papers  (2022-05-31T17:03:50Z)
• Mukayese: Turkish NLP Strikes Back [0.19116784879310023]
  We demonstrate that languages such as Turkish are left behind the state-of-the-art in NLP applications. We present Mukayese, a set of NLP benchmarks for the Turkish language. We present four new benchmarking datasets in Turkish for language modeling, sentence segmentation, and spell checking.
  arXiv  Detail & Related papers  (2022-03-02T16:18:44Z)
• Toward More Meaningful Resources for Lower-resourced Languages [2.3513645401551333]
  We examine the contents of the names stored in Wikidata for a few lower-resourced languages. We discuss quality issues present in WikiAnn and evaluate whether it is a useful supplement to hand annotated data. We conclude with recommended guidelines for resource development.
  arXiv  Detail & Related papers  (2022-02-24T18:39:57Z)
• Low-resource Languages: A Review of Past Work and Future Challenges [68.8204255655161]
  A current problem in NLP is massaging and processing low-resource languages which lack useful training attributes such as supervised data, number of native speakers or experts, etc. This review paper concisely summarizes previous groundbreaking achievements made towards resolving this problem, and analyzes potential improvements in the context of the overall future research direction.
  arXiv  Detail & Related papers  (2020-06-12T15:21:57Z)
• Improving Candidate Generation for Low-resource Cross-lingual Entity Linking [81.41804263432684]
  Cross-lingual entity linking (XEL) is the task of finding referents in a target-language knowledge base (KB) for mentions extracted from source-language texts. In this paper, we propose three improvements that (1) reduce the disconnect between entity mentions and KB entries, and (2) improve the robustness of the model to low-resource scenarios.
  arXiv  Detail & Related papers  (2020-03-03T05:32:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.