The Generator-Eraser Paradox: Community Guidelines for Responsible LLM-Assisted Dialect Resource Creation

• URL: http://arxiv.org/abs/2606.06004v1
• Date: Thu, 04 Jun 2026 10:57:36 GMT
• Title: The Generator-Eraser Paradox: Community Guidelines for Responsible LLM-Assisted Dialect Resource Creation
• Abstract summary: Dialect resources occupy a unique position at the intersection of scientific description, cultural preservation, and computational infrastructure.<n>Large language models offer powerful capabilities for accelerating dialect resource development through retrieval-grounded drafting, corpus navigation, metadata enrichment, and annotation workflow support.<n>However, the same systems pose substantial risks: they can contribute to dialect erasure by privileging prestige varieties, homogenizing orthography, and enabling synthetic feedback loops that reduce linguistic diversity over time.
• Score: 0.7734073750595959
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Dialect resources occupy a unique position at the intersection of scientific description, cultural preservation, and computational infrastructure. Large language models offer powerful capabilities for accelerating dialect resource development through retrieval-grounded drafting, corpus navigation, metadata enrichment, and annotation workflow support. However, the same systems pose substantial risks: they can contribute to dialect erasure by privileging prestige varieties, homogenizing orthography, and enabling synthetic feedback loops that reduce linguistic diversity over time. These risks are particularly acute for language varieties characterized by diglossia, limited written standardization, or marginalized speaker communities. This paper makes three contributions. First, we integrate insights from variationist sociolinguistics and corpus linguistics to formalize the generator-eraser paradox as a theoretical framework for understanding the dual nature of LLM-assisted dialect work. Second, we derive 12 community guidelines that operationalize this framework into implementable design requirements for dialect resource creation and documentation. Third, we provide an in-depth case study of Arabic dialects, including a structured comparison of widely used resources, to demonstrate how these guidelines address language-specific challenges including diglossia, orthographic variability, and community governance. The contribution is conceptual and operational rather than experimental, with the goal of enabling dialect communities and resource builders across languages to adopt LLMs without sacrificing authenticity, variation, or sovereignty.

Related papers

• Can Dialects Be Steered Like Languages? Sparse Neurons and Distributed Directions in Arabic LLMs [60.9510933247844]
  Key challenge in Arabic NLP is the scarcity of dialectal data relative to Modern Standard Arabic (MSA)<n>From an interpretability perspective, this raises a fundamental question: where and how are dialectal features encoded within model internals?<n>This study investigates two complementary inference-time approaches that serve simultaneously as interpretability probes and control mechanisms.
  arXiv  Detail & Related papers  (2026-07-04T16:04:44Z)
• LANG: Reinforcement Learning for Multilingual Reasoning with Language-Adaptive Hint Guidance [77.58408743830314]
  Reinforcement learning has proven effective for enhancing multi-step reasoning in large language models.<n>But its benefits have not fully translated to multilingual contexts.<n>We develop a novel framework that leverages language-conditioned hints to guide exploration in non-English reasoning tasks.
  arXiv  Detail & Related papers  (2026-05-21T14:47:52Z)
• Sign Language Recognition and Translation for Low-Resource Languages: Challenges and Pathways Forward [0.0]
  Sign languages are natural, visual-gestural languages used by Deaf communities worldwide.<n>Over 300 distinct sign languages remain severely low-resource due to limited documentation, sparse datasets, and insufficient computational tools.<n>This systematic review synthesizes literature on sign language recognition and translation for under-resourced languages, using Azerbaijan Sign Language (AzSL) as a case study.
  arXiv  Detail & Related papers  (2026-05-12T13:13:56Z)
• Shaping Schema via Language Representation as the Next Frontier for LLM Intelligence Expanding [88.11781604392606]
  This paper defines language representation as the linguistic and symbolic constructs used to map and model the real world.<n>We argue that shaping schemas through advanced language representation is the next frontier for expanding Large Language Models.
  arXiv  Detail & Related papers  (2026-05-10T02:42:29Z)
• IDIOLEX: Unified and Continuous Representations for Idiolectal and Stylistic Variation [88.98544786373212]
  Existing sentence representations primarily encode what a sentence says, rather than how it is expressed.<n>We develop sentence representations that capture style and dialect, decoupled from semantic content.<n>We introduce IDIOLEX, a framework for training models that combines supervision from a sentence's provenance with linguistic features of a sentence's content.
  arXiv  Detail & Related papers  (2026-04-06T14:17:24Z)
• \textit{Versteasch du mi?} Computational and Socio-Linguistic Perspectives on GenAI, LLMs, and Non-Standard Language [2.6461502412805062]
  Large Language Models and generative artificial intelligence have been shown to be "unfair" to less-spoken languages and to deepen the digital language divide.<n>This paper argues that these technologies exacerbate epistemologies of language as "monolithic, monolingual, syntactically standardized systems of meaning"<n>We take two different complexes of non-standard linguistic varieties in our respective repertoires--South Tyrolean dialects, which are widely used in informal communication in South Tyrol, Italy, as well as varieties of Kurdish--as starting points to an interdisciplinary exploration of the intersections between GenAI and linguistic variation and standardisation.
  arXiv  Detail & Related papers  (2026-03-30T09:34:41Z)
• Integrating Linguistics and AI: Morphological Analysis and Corpus development of Endangered Toto Language of West Bengal [0.6089496237595778]
  This paper is a part of a project which aims to develop a trilingual (Toto-Bangla-English) language learning application.<n>It aims to digitally archive and promote the endangered Toto language of West Bengal, India.
  arXiv  Detail & Related papers  (2025-10-26T11:22:46Z)
• The Shrinking Landscape of Linguistic Diversity in the Age of Large Language Models [7.811355338367627]
  We show that the widespread adoption of large language models (LLMs) as writing assistants is linked to notable declines in linguistic diversity.<n>We show that while the core content of texts is retained when LLMs polish and rewrite texts, not only do they homogenize writing styles, but they also alter stylistic elements in a way that selectively amplifies certain dominant characteristics or biases while suppressing others.
  arXiv  Detail & Related papers  (2025-02-16T20:51:07Z)
• LIMBA: An Open-Source Framework for the Preservation and Valorization of Low-Resource Languages using Generative Models [62.47865866398233]
  This white paper proposes a framework to generate linguistic tools for low-resource languages. By addressing the data scarcity that hinders intelligent applications for such languages, we contribute to promoting linguistic diversity.
  arXiv  Detail & Related papers  (2024-11-20T16:59:41Z)
• Lens: Rethinking Multilingual Enhancement for Large Language Models [70.85065197789639]
  We propose Lens, a novel approach to enhance multilingual capabilities in large language models (LLMs)<n>Lens operates on two subspaces: the language-agnostic subspace, where it aligns target languages with the central language to inherit strong semantic representations, and the language-specific subspace, where it separates target and central languages to preserve linguistic specificity.<n>Lens significantly improves multilingual performance while maintaining the model's English proficiency, achieving better results with less computational cost compared to existing post-training approaches.
  arXiv  Detail & Related papers  (2024-10-06T08:51:30Z)
• Are Structural Concepts Universal in Transformer Language Models? Towards Interpretable Cross-Lingual Generalization [27.368684663279463]
  We investigate the potential for explicitly aligning conceptual correspondence between languages to enhance cross-lingual generalization. Using the syntactic aspect of language as a testbed, our analyses of 43 languages reveal a high degree of alignability. We propose a meta-learning-based method to learn to align conceptual spaces of different languages.
  arXiv  Detail & Related papers  (2023-10-19T14:50:51Z)
• XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning [68.57658225995966]
  Cross-lingual Choice of Plausible Alternatives (XCOPA) is a typologically diverse multilingual dataset for causal commonsense reasoning in 11 languages. We evaluate a range of state-of-the-art models on this novel dataset, revealing that the performance of current methods falls short compared to translation-based transfer.
  arXiv  Detail & Related papers  (2020-05-01T12:22:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.