Digital Linguistic Bias in Spanish: Evidence from Lexical Variation in LLMs

• URL: http://arxiv.org/abs/2602.09346v1
• Date: Tue, 10 Feb 2026 02:42:22 GMT
• Title: Digital Linguistic Bias in Spanish: Evidence from Lexical Variation in LLMs
• Authors: Yoshifumi Kawasaki,
• Abstract summary: This study examines the extent to which Large Language Models (LLMs) capture geographic lexical variation in Spanish.<n>Treating LLMs as virtual informants, we probe their dialectal knowledge using two survey-style question formats: Yes-No questions and multiple-choice questions.<n>Our evaluation covers more than 900 lexical items across 21 Spanish-speaking countries and is conducted at both the country and dialectal area levels.
• Score: 0.4771833920251869
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This study examines the extent to which Large Language Models (LLMs) capture geographic lexical variation in Spanish, a language that exhibits substantial regional variation. Treating LLMs as virtual informants, we probe their dialectal knowledge using two survey-style question formats: Yes-No questions and multiple-choice questions. To this end, we exploited a large-scale, expert-curated database of Spanish lexical variation. Our evaluation covers more than 900 lexical items across 21 Spanish-speaking countries and is conducted at both the country and dialectal area levels. Across both evaluation formats, the results reveal systematic differences in how LLMs represent Spanish language varieties. Lexical variation associated with Spain, Equatorial Guinea, Mexico & Central America, and the La Plata River is recognized more accurately by the models, while the Chilean variety proves particularly difficult for the models to distinguish. Importantly, differences in the volume of country-level digital resources do not account for these performance patterns, suggesting that factors beyond data quantity shape dialectal representation in LLMs. By providing a fine-grained, large-scale evaluation of geographic lexical variation, this work advances empirical understanding of dialectal knowledge in LLMs and contributes new evidence to discussions of Digital Linguistic Bias in Spanish.

Related papers

• Do You Know About My Nation? Investigating Multilingual Language Models' Cultural Literacy Through Factual Knowledge [68.6805229085352]
  Most multilingual question-answering benchmarks do not factor in regional diversity in the information they capture.<n>XNationQA encompasses a total of 49,280 questions on the geography, culture, and history of nine countries, presented in seven languages.<n>We benchmark eight standard multilingual LLMs on XNationQA and evaluate them using two novel transference metrics.
  arXiv  Detail & Related papers  (2025-11-01T18:41:34Z)
• Lost in Variation? Evaluating NLI Performance in Basque and Spanish Geographical Variants [7.160574787275442]
  We evaluate the capacity of current language technologies to understand Basque and Spanish language varieties.<n>We use Natural Language Inference (NLI) as a pivot task and introduce a novel, manually-curated parallel dataset.
  arXiv  Detail & Related papers  (2025-06-18T08:20:19Z)
• Disparities in LLM Reasoning Accuracy and Explanations: A Case Study on African American English [66.97110551643722]
  We investigate dialectal disparities in Large Language Models (LLMs) reasoning tasks.<n>We find that LLMs produce less accurate responses and simpler reasoning chains and explanations for AAE inputs.<n>These findings highlight systematic differences in how LLMs process and reason about different language varieties.
  arXiv  Detail & Related papers  (2025-03-06T05:15:34Z)
• Large Language Models Reflect the Ideology of their Creators [71.65505524599888]
  Large language models (LLMs) are trained on vast amounts of data to generate natural language.<n>This paper shows that the ideological stance of an LLM appears to reflect the worldview of its creators.
  arXiv  Detail & Related papers  (2024-10-24T04:02:30Z)
• Beneath the Surface of Consistency: Exploring Cross-lingual Knowledge Representation Sharing in LLMs [31.893686987768742]
  Language models are inconsistent in their ability to answer the same factual question across languages. We explore multilingual factual knowledge through two aspects: the model's ability to answer a query consistently across languages, and the ability to ''store'' answers in a shared representation for several languages.
  arXiv  Detail & Related papers  (2024-08-20T08:38:30Z)
• Multi-FAct: Assessing Factuality of Multilingual LLMs using FActScore [14.91669562846729]
  We introduce a simple pipeline for multilingual factuality evaluation, by applying FActScore for diverse languages. We evaluate the factual accuracy of long-form text generation in topics that reflect regional diversity.
  arXiv  Detail & Related papers  (2024-02-28T04:43:46Z)
• Quantifying the Dialect Gap and its Correlates Across Languages [69.18461982439031]
  This work will lay the foundation for furthering the field of dialectal NLP by laying out evident disparities and identifying possible pathways for addressing them through mindful data collection.
  arXiv  Detail & Related papers  (2023-10-23T17:42:01Z)
• Cross-Lingual Knowledge Editing in Large Language Models [73.12622532088564]
  Knowledge editing has been shown to adapt large language models to new knowledge without retraining from scratch. It is still unknown the effect of source language editing on a different target language. We first collect a large-scale cross-lingual synthetic dataset by translating ZsRE from English to Chinese.
  arXiv  Detail & Related papers  (2023-09-16T11:07:52Z)
• Don't Trust ChatGPT when Your Question is not in English: A Study of Multilingual Abilities and Types of LLMs [16.770697902481107]
  Large Language Models (LLMs) have demonstrated exceptional natural language understanding abilities. We propose a systematic way of qualifying the performance disparities of LLMs under multilingual settings. The results show that GPT exhibits highly translating-like behaviour in multilingual settings.
  arXiv  Detail & Related papers  (2023-05-24T02:05:03Z)
• A large scale lexical and semantic analysis of Spanish language variations in Twitter [2.3511629321667096]
  This manuscript presents a broad analysis describing lexical and semantic relationships among 26 Spanish-speaking countries around the globe. We analyze four-year of the Twitter geotagged public stream to provide an extensive survey of the Spanish language vocabularies of different countries.
  arXiv  Detail & Related papers  (2021-10-12T16:21:03Z)
• Probing Pretrained Language Models for Lexical Semantics [76.73599166020307]
  We present a systematic empirical analysis across six typologically diverse languages and five different lexical tasks. Our results indicate patterns and best practices that hold universally, but also point to prominent variations across languages and tasks.
  arXiv  Detail & Related papers  (2020-10-12T14:24:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.