Enhancing NER Performance in Low-Resource Pakistani Languages using Cross-Lingual Data Augmentation

• URL: http://arxiv.org/abs/2504.08792v1
• Date: Mon, 07 Apr 2025 15:18:34 GMT
• Title: Enhancing NER Performance in Low-Resource Pakistani Languages using Cross-Lingual Data Augmentation
• Authors: Toqeer Ehsan, Thamar Solorio,
• Abstract summary: We propose a data augmentation technique that generates culturally plausible sentences and experiments on four low-resource Pakistani languages.<n>By fine-tuning multilingual masked Large Language Models (LLMs), our approach demonstrates significant improvements in NER performance for Shahmukhi and Pashto.
• Score: 7.383944919243126
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Named Entity Recognition (NER), a fundamental task in Natural Language Processing (NLP), has shown significant advancements for high-resource languages. However, due to a lack of annotated datasets and limited representation in Pre-trained Language Models (PLMs), it remains understudied and challenging for low-resource languages. To address these challenges, we propose a data augmentation technique that generates culturally plausible sentences and experiments on four low-resource Pakistani languages; Urdu, Shahmukhi, Sindhi, and Pashto. By fine-tuning multilingual masked Large Language Models (LLMs), our approach demonstrates significant improvements in NER performance for Shahmukhi and Pashto. We further explore the capability of generative LLMs for NER and data augmentation using few-shot learning.

Related papers

• Generative AI for Named Entity Recognition in Low-Resource Language Nepali [0.0]
  This paper investigates the application of Large Language Models (LLMs) for Named Entity Recognition (NER) in Nepali.<n>LLMs are especially promising for low-resource languages due to their ability to learn from limited data.<n>Our results offer valuable contributions to the advancement of NLP research in languages like Nepali.
  arXiv  Detail & Related papers  (2025-03-12T20:40:09Z)
• Cross-Lingual Transfer for Low-Resource Natural Language Processing [0.32634122554914]
  Cross-lingual transfer learning is a research area aimed at leveraging data and models from high-resource languages to improve NLP performance.<n>This thesis presents a new method to improve data-based transfer with T-Projection, a state-of-the-art annotation projection method.<n>For model-based transfer, we introduce a constrained decoding algorithm that enhances cross-lingual Sequence Labeling in zero-shot settings.<n>Finally, we develop Medical mT5, the first multilingual text-to-text medical model.
  arXiv  Detail & Related papers  (2025-02-04T21:17:46Z)
• Enhancing Code Generation for Low-Resource Languages: No Silver Bullet [55.39571645315926]
  Large Language Models (LLMs) rely on large and diverse datasets to learn syntax, semantics, and usage patterns of programming languages.<n>For low-resource languages, the limited availability of such data hampers the models' ability to generalize effectively.<n>We present an empirical study investigating the effectiveness of several approaches for boosting LLMs' performance on low-resource languages.
  arXiv  Detail & Related papers  (2025-01-31T12:23:28Z)
• LIMBA: An Open-Source Framework for the Preservation and Valorization of Low-Resource Languages using Generative Models [62.47865866398233]
  This white paper proposes a framework to generate linguistic tools for low-resource languages. By addressing the data scarcity that hinders intelligent applications for such languages, we contribute to promoting linguistic diversity.
  arXiv  Detail & Related papers  (2024-11-20T16:59:41Z)
• Do Large Language Models Speak All Languages Equally? A Comparative Study in Low-Resource Settings [12.507989493130175]
  Large language models (LLMs) have garnered significant interest in natural language processing (NLP) Recent studies have highlighted the limitations of LLMs in low-resource languages. We present datasets for sentiment and hate speech tasks by translating from English to Bangla, Hindi, and Urdu.
  arXiv  Detail & Related papers  (2024-08-05T05:09:23Z)
• MoE-CT: A Novel Approach For Large Language Models Training With Resistance To Catastrophic Forgetting [53.77590764277568]
  We introduce a novel MoE-CT architecture that separates the base model's learning from the multilingual expansion process. Our design freezes the original LLM parameters, thus safeguarding its performance in high-resource languages, while an appended MoE module, trained on diverse language datasets, augments low-resource language proficiency.
  arXiv  Detail & Related papers  (2024-06-25T11:03:45Z)
• Bridging the Bosphorus: Advancing Turkish Large Language Models through Strategies for Low-Resource Language Adaptation and Benchmarking [1.3716808114696444]
  Large Language Models (LLMs) are becoming crucial across various fields, emphasizing the urgency for high-quality models in underrepresented languages. This study explores the unique challenges faced by low-resource languages, such as data scarcity, model selection, evaluation, and computational limitations.
  arXiv  Detail & Related papers  (2024-05-07T21:58:45Z)
• Transferring BERT Capabilities from High-Resource to Low-Resource Languages Using Vocabulary Matching [1.746529892290768]
  This work presents a novel approach to transfer BERT capabilities from high-resource to low-resource languages using vocabulary matching. We conduct experiments on the Silesian and Kashubian languages and demonstrate the effectiveness of our approach to improve the performance of BERT models even when the target language has minimal training data.
  arXiv  Detail & Related papers  (2024-02-22T09:49:26Z)
• Enhancing Multilingual Capabilities of Large Language Models through Self-Distillation from Resource-Rich Languages [60.162717568496355]
  Large language models (LLMs) have been pre-trained on multilingual corpora. Their performance still lags behind in most languages compared to a few resource-rich languages.
  arXiv  Detail & Related papers  (2024-02-19T15:07:32Z)
• Democratizing LLMs for Low-Resource Languages by Leveraging their English Dominant Abilities with Linguistically-Diverse Prompts [75.33019401706188]
  Large language models (LLMs) are known to effectively perform tasks by simply observing few exemplars. We propose to assemble synthetic exemplars from a diverse set of high-resource languages to prompt the LLMs to translate from any language into English. Our unsupervised prompting method performs on par with supervised few-shot learning in LLMs of different sizes for translations between English and 13 Indic and 21 African low-resource languages.
  arXiv  Detail & Related papers  (2023-06-20T08:27:47Z)
• Reinforced Iterative Knowledge Distillation for Cross-Lingual Named Entity Recognition [54.92161571089808]
  Cross-lingual NER transfers knowledge from rich-resource language to languages with low resources. Existing cross-lingual NER methods do not make good use of rich unlabeled data in target languages. We develop a novel approach based on the ideas of semi-supervised learning and reinforcement learning.
  arXiv  Detail & Related papers  (2021-06-01T05:46:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.