Is It Navajo? Accurate Language Detection in Endangered Athabaskan Languages

• URL: http://arxiv.org/abs/2501.15773v2
• Date: Mon, 10 Feb 2025 22:42:15 GMT
• Title: Is It Navajo? Accurate Language Detection in Endangered Athabaskan Languages
• Authors: Ivory Yang, Weicheng Ma, Chunhui Zhang, Soroush Vosoughi,
• Abstract summary: Endangered languages, such as Navajo, are significantly underrepresented in contemporary language technologies. This study evaluates Google's Language Identification (LangID) tool, which does not currently support any Native American languages.
• Score: 34.78841410279943
• License:
• Abstract: Endangered languages, such as Navajo - the most widely spoken Native American language - are significantly underrepresented in contemporary language technologies, exacerbating the challenges of their preservation and revitalization. This study evaluates Google's Language Identification (LangID) tool, which does not currently support any Native American languages. To address this, we introduce a random forest classifier trained on Navajo and twenty erroneously suggested languages by LangID. Despite its simplicity, the classifier achieves near-perfect accuracy (97-100%). Additionally, the model demonstrates robustness across other Athabaskan languages - a family of Native American languages spoken primarily in Alaska, the Pacific Northwest, and parts of the Southwestern United States - suggesting its potential for broader application. Our findings underscore the pressing need for NLP systems that prioritize linguistic diversity and adaptability over centralized, one-size-fits-all solutions, especially in supporting underrepresented languages in a multicultural world. This work directly contributes to ongoing efforts to address cultural biases in language models and advocates for the development of culturally localized NLP tools that serve diverse linguistic communities.

Related papers

• Indigenous Languages Spoken in Argentina: A Survey of NLP and Speech Resources [45.07333085270152]
  Argentina has a large yet little-known Indigenous linguistic diversity, encompassing at least 40 different languages. We present a systematization of the Indigenous languages spoken in Argentina, classifying them into seven language families. For each one, we present an estimation of the national Indigenous population size, based on the most recent Argentinian census.
  arXiv  Detail & Related papers  (2025-01-17T03:47:19Z)
• NusaBERT: Teaching IndoBERT to be Multilingual and Multicultural [0.0]
  NusaBERT builds upon IndoBERT by incorporating vocabulary expansion and leveraging a diverse multilingual corpus that includes regional languages and dialects. Through rigorous evaluation across a range of benchmarks, NusaBERT demonstrates state-of-the-art performance in tasks involving multiple languages of Indonesia.
  arXiv  Detail & Related papers  (2024-03-04T08:05:34Z)
• What Do Dialect Speakers Want? A Survey of Attitudes Towards Language Technology for German Dialects [60.8361859783634]
  We survey speakers of dialects and regional languages related to German. We find that respondents are especially in favour of potential NLP tools that work with dialectal input.
  arXiv  Detail & Related papers  (2024-02-19T09:15:28Z)
• NusaWrites: Constructing High-Quality Corpora for Underrepresented and Extremely Low-Resource Languages [54.808217147579036]
  We conduct a case study on Indonesian local languages. We compare the effectiveness of online scraping, human translation, and paragraph writing by native speakers in constructing datasets. Our findings demonstrate that datasets generated through paragraph writing by native speakers exhibit superior quality in terms of lexical diversity and cultural content.
  arXiv  Detail & Related papers  (2023-09-19T14:42:33Z)
• Multi-lingual and Multi-cultural Figurative Language Understanding [69.47641938200817]
  Figurative language permeates human communication, but is relatively understudied in NLP. We create a dataset for seven diverse languages associated with a variety of cultures: Hindi, Indonesian, Javanese, Kannada, Sundanese, Swahili and Yoruba. Our dataset reveals that each language relies on cultural and regional concepts for figurative expressions, with the highest overlap between languages originating from the same region. All languages exhibit a significant deficiency compared to English, with variations in performance reflecting the availability of pre-training and fine-tuning data.
  arXiv  Detail & Related papers  (2023-05-25T15:30:31Z)
• How can NLP Help Revitalize Endangered Languages? A Case Study and Roadmap for the Cherokee Language [91.79339725967073]
  More than 43% of the languages spoken in the world are endangered. In this work, we focus on discussing how NLP can help revitalize endangered languages. We take Cherokee, a severely-endangered Native American language, as a case study.
  arXiv  Detail & Related papers  (2022-04-25T18:25:57Z)
• Role of Language Relatedness in Multilingual Fine-tuning of Language Models: A Case Study in Indo-Aryan Languages [34.79533646549939]
  We explore the impact of leveraging the relatedness of languages that belong to the same family in NLP models using multilingual fine-tuning. Low resource languages, such as Oriya and Punjabi, are found to be the largest beneficiaries of multilingual fine-tuning.
  arXiv  Detail & Related papers  (2021-09-22T06:37:39Z)
• Discovering Representation Sprachbund For Multilingual Pre-Training [139.05668687865688]
  We generate language representation from multilingual pre-trained models and conduct linguistic analysis. We cluster all the target languages into multiple groups and name each group as a representation sprachbund. Experiments are conducted on cross-lingual benchmarks and significant improvements are achieved compared to strong baselines.
  arXiv  Detail & Related papers  (2021-09-01T09:32:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.