Large language models and linguistic intentionality

• URL: http://arxiv.org/abs/2404.09576v2
• Date: Mon, 16 Sep 2024 08:35:51 GMT
• Title: Large language models and linguistic intentionality
• Authors: Jumbly Grindrod,
• Abstract summary: I will argue that we should instead consider whether language models meet the criteria given by our best metasemantic theories of linguistic content. I will argue that it is a mistake to think that the failure of LLMs to meet plausible conditions for mental intentionality thereby renders their outputs meaningless.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Do large language models like Chat-GPT or LLaMa meaningfully use the words they produce? Or are they merely clever prediction machines, simulating language use by producing statistically plausible text? There have already been some initial attempts to answer this question by showing that these models meet the criteria for entering meaningful states according to metasemantic theories of mental content. In this paper, I will argue for a different approach - that we should instead consider whether language models meet the criteria given by our best metasemantic theories of linguistic content. In that vein, I will illustrate how this can be done by applying two such theories to the case of language models: Gareth Evans' (1982) account of naming practices and Ruth Millikan's (1984, 2004, 2005) teleosemantics. In doing so, I will argue that it is a mistake to think that the failure of LLMs to meet plausible conditions for mental intentionality thereby renders their outputs meaningless, and that a distinguishing feature of linguistic intentionality - dependency on a pre-existing linguistic system - allows for the plausible result LLM outputs are meaningful.

Related papers

• Are LLMs good pragmatic speakers? [3.4113474745671923]
  Large language models (LLMs) are trained on data assumed to include natural language pragmatics, but do they actually behave like pragmatic speakers? We attempt to answer this question using the Rational Speech Act (RSA) framework, which models pragmatic reasoning in human communication. We find that while scores from the LLM have some positive correlation with those from RSA, there isn't sufficient evidence to claim that it behaves like a pragmatic speaker.
  arXiv  Detail & Related papers  (2024-11-03T13:23:18Z)
• Are LLMs Models of Distributional Semantics? A Case Study on Quantifiers [14.797001158310092]
  We argue that distributional semantics models struggle with truth-conditional reasoning and symbolic processing. Contrary to expectations, we find that LLMs align more closely with human judgements on exact quantifiers versus vague ones.
  arXiv  Detail & Related papers  (2024-10-17T19:28:35Z)
• Kallini et al. (2024) do not compare impossible languages with constituency-based ones [0.0]
  A central goal of linguistic theory is to find a characterization of the notion "possible human language" Recent large language models (LLMs) in NLP applications arguably raises the possibility that LLMs might be computational devices that meet this goal. I explain the confound and suggest some ways forward towards constructing a comparison that appropriately tests the underlying issue.
  arXiv  Detail & Related papers  (2024-10-16T06:16:30Z)
• One Language, Many Gaps: Evaluating Dialect Fairness and Robustness of Large Language Models in Reasoning Tasks [55.35278531907263]
  We present the first study on Large Language Models' fairness and robustness to a dialect in canonical reasoning tasks. We hire AAVE speakers to rewrite seven popular benchmarks, such as HumanEval and GSM8K. We find that, compared to Standardized English, almost all of these widely used models show significant brittleness and unfairness to queries in AAVE.
  arXiv  Detail & Related papers  (2024-10-14T18:44:23Z)
• A Peek into Token Bias: Large Language Models Are Not Yet Genuine Reasoners [58.15511660018742]
  This study introduces a hypothesis-testing framework to assess whether large language models (LLMs) possess genuine reasoning abilities. We develop carefully controlled synthetic datasets, featuring conjunction fallacy and syllogistic problems.
  arXiv  Detail & Related papers  (2024-06-16T19:22:53Z)
• LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Models [52.03659714625452]
  Recently developed large language models (LLMs) have been shown to perform remarkably well on a wide range of language understanding tasks. But, can they really "reason" over the natural language? This question has been receiving significant research attention and many reasoning skills such as commonsense, numerical, and qualitative have been studied.
  arXiv  Detail & Related papers  (2024-04-23T21:08:49Z)
• Fantastic Semantics and Where to Find Them: Investigating Which Layers of Generative LLMs Reflect Lexical Semantics [50.982315553104975]
  We investigate the bottom-up evolution of lexical semantics for a popular large language model, namely Llama2. Our experiments show that the representations in lower layers encode lexical semantics, while the higher layers, with weaker semantic induction, are responsible for prediction. This is in contrast to models with discriminative objectives, such as mask language modeling, where the higher layers obtain better lexical semantics.
  arXiv  Detail & Related papers  (2024-03-03T13:14:47Z)
• Evaluating Gender Bias in Large Language Models via Chain-of-Thought Prompting [87.30837365008931]
  Large language models (LLMs) equipped with Chain-of-Thought (CoT) prompting are able to make accurate incremental predictions even on unscalable tasks. This study examines the impact of LLMs' step-by-step predictions on gender bias in unscalable tasks.
  arXiv  Detail & Related papers  (2024-01-28T06:50:10Z)
• Why Can Large Language Models Generate Correct Chain-of-Thoughts? [10.888196404348093]
  We introduce a two-level hierarchical graphical model tailored for natural language generation. We establish a compelling geometrical convergence rate that gauges the likelihood of an LLM-generated chain of thoughts.
  arXiv  Detail & Related papers  (2023-10-20T15:09:46Z)
• Towards Explainable and Language-Agnostic LLMs: Symbolic Reverse Engineering of Language at Scale [0.0]
  Large language models (LLMs) have achieved a milestone that undenia-bly changed many held beliefs in artificial intelligence (AI) We argue for a bottom-up reverse engineering of language in a symbolic setting.
  arXiv  Detail & Related papers  (2023-05-30T15:15:40Z)
• Transparency Helps Reveal When Language Models Learn Meaning [71.96920839263457]
  Our systematic experiments with synthetic data reveal that, with languages where all expressions have context-independent denotations, both autoregressive and masked language models learn to emulate semantic relations between expressions. Turning to natural language, our experiments with a specific phenomenon -- referential opacity -- add to the growing body of evidence that current language models do not well-represent natural language semantics.
  arXiv  Detail & Related papers  (2022-10-14T02:35:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.