Chatbots Output Meaningful (but Problematic) Language

• URL: http://arxiv.org/abs/2606.02973v1
• Date: Tue, 02 Jun 2026 00:24:42 GMT
• Title: Chatbots Output Meaningful (but Problematic) Language
• Abstract summary: Most ordinary users, as well as AI engineers, take the answer to be trivially "yes"<n>But many cognitive scientists, linguists, and philosophers of language argue that dominant intentionalist accounts of language and meaning deliver the opposite conclusion.<n>Theorists more sympathetic to ordinary users' intuitions have therefore advocated a radical "de-anthropomorphization" of language.
• Score: 1.246150324257064
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Are utterances by AI chatbots meaningful? Concretely, if a user asks, say, Anthropic's agent Claude, "What is the capital of Spain?" and Claude answers, "Madrid is the capital of Spain," does that sentence have its ordinary meaning -- and does it express a true proposition? Most ordinary users, as well as AI engineers, take the answer to be trivially "yes." However, many cognitive scientists, linguists, and philosophers of language argue that dominant intentionalist accounts of language and meaning deliver the opposite conclusion. Theorists more sympathetic to ordinary users' intuitions have therefore advocated a radical "de-anthropomorphization" of language, revising our understanding of mental states, intentions, and semantic content to capture the intuition that the outputs of LLMs are meaningful. We take a different approach. While we, too, argue that LLM outputs are meaningful, we contend that a proper theory of human language already applies, as is, to current chatbots. Meaning is a low bar: claiming that LLM outputs are meaningful does not require positing mental states, intentions, rationality, or the cognitive capacities requisite for communication in LLMs -- or, indeed, making any other anthropomorphic assumptions. People do have communicative intentions (typically successful ones), but nevertheless, even in humans, language production can depart from what the speaker has in mind. Our view has important consequences for how we should theorize about -- and critically engage with -- both human linguistic output and synthetically generated text. In particular, to say that chatbots produce meaningful text is not by any means to endorse what they output, or to assume that the technology is (or is not) good, powerful, appropriate, or useful.

Related papers

• Some hypotheses on how chatbots work in problem-solving-driven conversations. Large Language Models as confirmation of the Innovation Illusion [0.0]
  Our argument draws on Cognitive Linguistics, Neuropsychology and Psychology.<n>Basic chatbots are assumed to consist of a Large Language Model (LLM) with a simple interface.<n>The main results are: a description of human understanding and thinking based on so-called metaphorical problem propagations.
  arXiv  Detail & Related papers  (2026-06-05T16:04:39Z)
• Tracing the ongoing emergence of human-like reasoning in Large Language Models [0.13048920509133807]
  Large Language Models show human-like performance on many tasks.<n>It remains unclear whether they reason like humans.<n>We find that humans enrich logical reasoning through pragmatic inferences across languages.
  arXiv  Detail & Related papers  (2026-05-20T15:28:52Z)
• On the Same Wavelength? Evaluating Pragmatic Reasoning in Language Models across Broad Concepts [69.69818198773244]
  We study a range of LMs on both language comprehension and language production.<n>We find that state-of-the-art LMs, but not smaller ones, achieve strong performance on language comprehension.
  arXiv  Detail & Related papers  (2025-09-08T17:59:32Z)
• On the Thinking-Language Modeling Gap in Large Language Models [68.83670974539108]
  We show that there is a significant gap between the modeling of languages and thoughts.<n>We propose a new prompt technique termed Language-of-Thoughts (LoT) to demonstrate and alleviate this gap.
  arXiv  Detail & Related papers  (2025-05-19T09:31:52Z)
• Intermediate Languages Matter: Formal Choice Drives Neurosymbolic LLM Reasoning [50.99811144731619]
  We show that the choice of formal language affects both the syntactic and the semantic reasoning capability.<n>We conclude that on average, context-aware encodings help LLMs to reason, while there is no apparent effect of using comments or markdown syntax.
  arXiv  Detail & Related papers  (2025-02-24T14:49:52Z)
• Trying to be human: Linguistic traces of stochastic empathy in language models [0.2638512174804417]
  Large language models (LLMs) are crucial drivers behind the increased quality of computer-generated content. Our work tests how two important factors contribute to the human vs AI race: empathy and an incentive to appear human.
  arXiv  Detail & Related papers  (2024-10-02T15:46:40Z)
• Social AI and The Equation of Wittgenstein's Language User With Calvino's Literature Machine [0.0]
  Is it sensical to ascribe psychological predicates to AI systems like chatbots based on large language models (LLMs)? Social AIs are not full-blown language users, but rather more like Italo Calvino's literature machines. The framework of mortal computation is used to show that social AIs lack the basic autopoiesis needed for narrative faccons de parler.
  arXiv  Detail & Related papers  (2024-05-23T09:51:44Z)
• Can Language Models Be Tricked by Language Illusions? Easier with Syntax, Harder with Semantics [7.759719313292493]
  We investigate LMs' more subtle judgments associated with "language illusions" We look at three illusions: the comparative illusion, the depth-charge illusion, and the negative polarity item (NPI) illusion. We show that LMs are limited both in their construal as cognitive models of human language processing and in their capacity to recognize nuanced but critical information in complicated language materials.
  arXiv  Detail & Related papers  (2023-11-02T16:44:24Z)
• The Neuro-Symbolic Inverse Planning Engine (NIPE): Modeling Probabilistic Social Inferences from Linguistic Inputs [50.32802502923367]
  We study the process of language driving and influencing social reasoning in a probabilistic goal inference domain. We propose a neuro-symbolic model that carries out goal inference from linguistic inputs of agent scenarios. Our model closely matches human response patterns and better predicts human judgements than using an LLM alone.
  arXiv  Detail & Related papers  (2023-06-25T19:38:01Z)
• Large Language Models are In-Context Semantic Reasoners rather than Symbolic Reasoners [75.85554779782048]
  Large Language Models (LLMs) have excited the natural language and machine learning community over recent years. Despite of numerous successful applications, the underlying mechanism of such in-context capabilities still remains unclear. In this work, we hypothesize that the learned textitsemantics of language tokens do the most heavy lifting during the reasoning process.
  arXiv  Detail & Related papers  (2023-05-24T07:33:34Z)
• Understanding Natural Language Understanding Systems. A Critical Analysis [91.81211519327161]
  The development of machines that guillemotlefttalk like usguillemotright, also known as Natural Language Understanding (NLU) systems, is the Holy Grail of Artificial Intelligence (AI) But never has the trust that we can build guillemotlefttalking machinesguillemotright been stronger than the one engendered by the last generation of NLU systems. Are we at the dawn of a new era, in which the Grail is finally closer to us?
  arXiv  Detail & Related papers  (2023-03-01T08:32:55Z)
• The Goldilocks of Pragmatic Understanding: Fine-Tuning Strategy Matters for Implicature Resolution by LLMs [26.118193748582197]
  We evaluate four categories of widely used state-of-the-art models. We find that, despite only evaluating on utterances that require a binary inference, models in three of these categories perform close to random. These results suggest that certain fine-tuning strategies are far better at inducing pragmatic understanding in models.
  arXiv  Detail & Related papers  (2022-10-26T19:04:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.