Language, Language Models, and What We're Talking About

• URL: http://arxiv.org/abs/2609.03577v1
• Date: Thu, 03 Sep 2026 09:27:19 GMT
• Title: Language, Language Models, and What We're Talking About
• Abstract summary: I want to bring attention to the nature of the systems which result from training and specialising models on translated and synthetic data.<n>I argue that this question cannot be answered if we do not first consider a clearer distinction between language models designed as technical products and language models designed as tools for studying language itself.
• Score: 17.406192329306247
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Language models are commonly discussed as technical artefacts, but they are obviously shaped by the linguistic worlds conveyed by data during their training. Using Italian language models as evidence, I want to bring attention to the nature of the systems which result from training and specialising models on translated and synthetic data, and further curating them, and to the meaning of testing them on equally unnatural data. Are these eventually models of Italian? Are they models of language? Does NLP still care about language? These questions yield another, more concrete question: what language do we actually want language models to produce? I argue that this question cannot be answered if we do not first consider a clearer distinction between language models designed as technical products and language models designed as tools for studying language itself. The answers then might be diverse, the languages we are talking about might be diverse, and the picture might not be as pessimistic as we fear.

Related papers

• Language Specific Knowledge: Do Models Know Better in X than in English? [9.923619418000488]
  Code-switching is a common phenomenon of alternating between different languages in the same utterance, thought, or conversation.<n>We coin the term Language Specific Knowledge (LSK) to represent this phenomenon.<n>We find that language models can perform better when using chain-of-thought reasoning in some languages other than English.
  arXiv  Detail & Related papers  (2025-05-21T00:31:13Z)
• How Linguistics Learned to Stop Worrying and Love the Language Models [17.413438037432414]
  We argue that the success of LMs obviates the need for studying linguistic theory and structure.<n>They force us to rethink arguments and ways of thinking that have been foundational in linguistics.<n>We offer an optimistic take on the relationship between language models and linguistics.
  arXiv  Detail & Related papers  (2025-01-28T16:13:19Z)
• Lost in Translation: Large Language Models in Non-English Content Analysis [0.0]
  Large language models have become the dominant approach for building AI systems to analyze and generate language online. Recently, researchers and technology companies have attempted to extend the capabilities of large language models into languages other than English.
  arXiv  Detail & Related papers  (2023-06-12T19:10:47Z)
• Neural Machine Translation for the Indigenous Languages of the Americas: An Introduction [102.13536517783837]
  Most languages from the Americas are among them, having a limited amount of parallel and monolingual data, if any. We discuss the recent advances and findings and open questions, product of an increased interest of the NLP community in these languages.
  arXiv  Detail & Related papers  (2023-06-11T23:27:47Z)
• Speaking Multiple Languages Affects the Moral Bias of Language Models [70.94372902010232]
  Pre-trained multilingual language models (PMLMs) are commonly used when dealing with data from multiple languages and cross-lingual transfer. Do the models capture moral norms from English and impose them on other languages? Our experiments demonstrate that, indeed, PMLMs encode differing moral biases, but these do not necessarily correspond to cultural differences or commonalities in human opinions.
  arXiv  Detail & Related papers  (2022-11-14T20:08:54Z)
• Transparency Helps Reveal When Language Models Learn Meaning [71.96920839263457]
  Our systematic experiments with synthetic data reveal that, with languages where all expressions have context-independent denotations, both autoregressive and masked language models learn to emulate semantic relations between expressions. Turning to natural language, our experiments with a specific phenomenon -- referential opacity -- add to the growing body of evidence that current language models do not well-represent natural language semantics.
  arXiv  Detail & Related papers  (2022-10-14T02:35:19Z)
• Do Multilingual Language Models Capture Differing Moral Norms? [71.52261949766101]
  Massively multilingual sentence representations are trained on large corpora of uncurated data. This may cause the models to grasp cultural values including moral judgments from the high-resource languages. The lack of data in certain languages can also lead to developing random and thus potentially harmful beliefs.
  arXiv  Detail & Related papers  (2022-03-18T12:26:37Z)
• Language Models are not Models of Language [0.0]
  Transfer learning has enabled large deep learning neural networks trained on the language modeling task to vastly improve performance. We argue that the term language model is misleading because deep learning models are not theoretical models of language.
  arXiv  Detail & Related papers  (2021-12-13T22:39:46Z)
• Towards Zero-shot Language Modeling [90.80124496312274]
  We construct a neural model that is inductively biased towards learning human languages. We infer this distribution from a sample of typologically diverse training languages. We harness additional language-specific side information as distant supervision for held-out languages.
  arXiv  Detail & Related papers  (2021-08-06T23:49:18Z)
• PIGLeT: Language Grounding Through Neuro-Symbolic Interaction in a 3D World [86.21137454228848]
  We factorize PIGLeT into a physical dynamics model, and a separate language model. PIGLeT can read a sentence, simulate neurally what might happen next, and then communicate that result through a literal symbolic representation. It is able to correctly forecast "what happens next" given an English sentence over 80% of the time, outperforming a 100x larger, text-to-text approach by over 10%.
  arXiv  Detail & Related papers  (2021-06-01T02:32:12Z)
• Should we Stop Training More Monolingual Models, and Simply Use Machine Translation Instead? [2.62121275102348]
  We show that machine translation is a mature technology, which raises a serious counter-argument for training native language models for low-resource languages. As English language models are improving at an unprecedented pace, which in turn improves machine translation, it is from an empirical and environmental stand-point more effective to translate data from low-resource languages into English.
  arXiv  Detail & Related papers  (2021-04-21T10:21:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.