When your Cousin has the Right Connections: Unsupervised Bilingual Lexicon Induction for Related Data-Imbalanced Languages

• URL: http://arxiv.org/abs/2305.14012v2
• Date: Mon, 25 Mar 2024 12:51:40 GMT
• Title: When your Cousin has the Right Connections: Unsupervised Bilingual Lexicon Induction for Related Data-Imbalanced Languages
• Authors: Niyati Bafna, Cristina España-Bonet, Josef van Genabith, Benoît Sagot, Rachel Bawden,
• Abstract summary: Unsupervised bilingual lexicon induction is most likely to be useful for low-resource languages, where large datasets are not available. We show that state-of-the-art BLI methods in the literature exhibit near-zero performance for severely data-imbalanced language pairs. We present a new method for unsupervised BLI between a related LRL and HRL that only requires inference on a masked language model of the HRL.
• Score: 29.346191691508125
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Most existing approaches for unsupervised bilingual lexicon induction (BLI) depend on good quality static or contextual embeddings requiring large monolingual corpora for both languages. However, unsupervised BLI is most likely to be useful for low-resource languages (LRLs), where large datasets are not available. Often we are interested in building bilingual resources for LRLs against related high-resource languages (HRLs), resulting in severely imbalanced data settings for BLI. We first show that state-of-the-art BLI methods in the literature exhibit near-zero performance for severely data-imbalanced language pairs, indicating that these settings require more robust techniques. We then present a new method for unsupervised BLI between a related LRL and HRL that only requires inference on a masked language model of the HRL, and demonstrate its effectiveness on truly low-resource languages Bhojpuri and Magahi (with <5M monolingual tokens each), against Hindi. We further present experiments on (mid-resource) Marathi and Nepali to compare approach performances by resource range, and release our resulting lexicons for five low-resource Indic languages: Bhojpuri, Magahi, Awadhi, Braj, and Maithili, against Hindi.

Related papers

• Machine Translation Hallucination Detection for Low and High Resource Languages using Large Language Models [12.447489454369636]
  This paper evaluates sentence-level hallucination detection approaches using Large Language Models (LLMs) and semantic similarity within massively multilingual embeddings. LLMs can achieve performance comparable or even better than previously proposed models, despite not being explicitly trained for any machine translation task.
  arXiv  Detail & Related papers  (2024-07-23T13:40:54Z)
• Crosslingual Capabilities and Knowledge Barriers in Multilingual Large Language Models [62.91524967852552]
  Large language models (LLMs) are typically multilingual due to pretraining on diverse multilingual corpora. But can these models relate corresponding concepts across languages, effectively being crosslingual? This study evaluates six state-of-the-art LLMs on inherently crosslingual tasks.
  arXiv  Detail & Related papers  (2024-06-23T15:15:17Z)
• GlotLID: Language Identification for Low-Resource Languages [51.38634652914054]
  GlotLID-M is an LID model that satisfies the desiderata of wide coverage, reliability and efficiency. It identifies 1665 languages, a large increase in coverage compared to prior work.
  arXiv  Detail & Related papers  (2023-10-24T23:45:57Z)
• On Bilingual Lexicon Induction with Large Language Models [81.6546357879259]
  We examine the potential of the latest generation of Large Language Models for the development of bilingual lexicons. We study 1) zero-shot prompting for unsupervised BLI and 2) few-shot in-context prompting with a set of seed translation pairs. Our work is the first to demonstrate strong BLI capabilities of text-to-text mLLMs.
  arXiv  Detail & Related papers  (2023-10-21T12:43:27Z)
• Democratizing LLMs for Low-Resource Languages by Leveraging their English Dominant Abilities with Linguistically-Diverse Prompts [75.33019401706188]
  Large language models (LLMs) are known to effectively perform tasks by simply observing few exemplars. We propose to assemble synthetic exemplars from a diverse set of high-resource languages to prompt the LLMs to translate from any language into English. Our unsupervised prompting method performs on par with supervised few-shot learning in LLMs of different sizes for translations between English and 13 Indic and 21 African low-resource languages.
  arXiv  Detail & Related papers  (2023-06-20T08:27:47Z)
• CharSpan: Utilizing Lexical Similarity to Enable Zero-Shot Machine Translation for Extremely Low-resource Languages [22.51558549091902]
  We address the task of machine translation (MT) from extremely low-resource language (ELRL) to English by leveraging cross-lingual transfer from 'closely-related' high-resource language (HRL) Many ELRLs share lexical similarities with some HRLs, which presents a novel modeling opportunity. Existing subword-based neural MT models do not explicitly harness this lexical similarity, as they only implicitly align HRL and ELRL latent embedding space. We propose a novel, CharSpan, approach based on 'character-span noise augmentation' into the training data of HRL. This serves as a
  arXiv  Detail & Related papers  (2023-05-09T07:23:01Z)
• Overlap-based Vocabulary Generation Improves Cross-lingual Transfer Among Related Languages [18.862296065737347]
  We argue that relatedness among languages in a language family along the dimension of lexical overlap may be leveraged to overcome some of the corpora limitations of LRLs. We propose Overlap BPE, a simple yet effective modification to the BPE vocabulary generation algorithm which enhances overlap across related languages.
  arXiv  Detail & Related papers  (2022-03-03T19:35:24Z)
• Exploiting Language Relatedness for Low Web-Resource Language Model Adaptation: An Indic Languages Study [14.34516262614775]
  We argue that relatedness among languages in a language family may be exploited to overcome some of the corpora limitations of LRLs. We focus on Indian languages, and exploit relatedness along two dimensions: (1) script (since many Indic scripts originated from the Brahmic script) and (2) sentence structure.
  arXiv  Detail & Related papers  (2021-06-07T20:43:02Z)
• Cross-lingual Machine Reading Comprehension with Language Branch Knowledge Distillation [105.41167108465085]
  Cross-lingual Machine Reading (CLMRC) remains a challenging problem due to the lack of large-scale datasets in low-source languages. We propose a novel augmentation approach named Language Branch Machine Reading (LBMRC) LBMRC trains multiple machine reading comprehension (MRC) models proficient in individual language. We devise a multilingual distillation approach to amalgamate knowledge from multiple language branch models to a single model for all target languages.
  arXiv  Detail & Related papers  (2020-10-27T13:12:17Z)
• Anchor-based Bilingual Word Embeddings for Low-Resource Languages [76.48625630211943]
  Good quality monolingual word embeddings (MWEs) can be built for languages which have large amounts of unlabeled text. MWEs can be aligned to bilingual spaces using only a few thousand word translation pairs. This paper proposes a new approach for building BWEs in which the vector space of the high resource source language is used as a starting point.
  arXiv  Detail & Related papers  (2020-10-23T19:17:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.