Reinterpreting 'the Company a Word Keeps': Towards Explainable and Ontologically Grounded Language Models

• URL: http://arxiv.org/abs/2406.06610v1
• Date: Thu, 6 Jun 2024 20:38:35 GMT
• Title: Reinterpreting 'the Company a Word Keeps': Towards Explainable and Ontologically Grounded Language Models
• Authors: Walid S. Saba,
• Abstract summary: We argue that the relative success of large language models (LLMs) is not a reflection on the symbolic vs. subsymbolic debate. We suggest employing the same successful bottom-up strategy employed in LLMs but in a symbolic setting.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We argue that the relative success of large language models (LLMs) is not a reflection on the symbolic vs. subsymbolic debate but a reflection on employing a successful bottom-up strategy of a reverse engineering of language at scale. However, and due to their subsymbolic nature whatever knowledge these systems acquire about language will always be buried in millions of weights none of which is meaningful on its own, rendering such systems utterly unexplainable. Furthermore, and due to their stochastic nature, LLMs will often fail in making the correct inferences in various linguistic contexts that require reasoning in intensional, temporal, or modal contexts. To remedy these shortcomings we suggest employing the same successful bottom-up strategy employed in LLMs but in a symbolic setting, resulting in explainable, language-agnostic, and ontologically grounded language models.

Related papers

• Large Models of What? Mistaking Engineering Achievements for Human Linguistic Agency [0.11510009152620666]
  We argue that claims regarding linguistic capabilities of Large Language Models (LLMs) are based on at least two unfounded assumptions. Language completeness assumes that a distinct and complete thing such as a natural language' exists. The assumption of data completeness relies on the belief that a language can be quantified and wholly captured by data.
  arXiv  Detail & Related papers  (2024-07-11T18:06:01Z)
• LLMs' Reading Comprehension Is Affected by Parametric Knowledge and Struggles with Hypothetical Statements [59.71218039095155]
  Task of reading comprehension (RC) provides a primary means to assess language models' natural language understanding (NLU) capabilities. If the context aligns with the models' internal knowledge, it is hard to discern whether the models' answers stem from context comprehension or from internal information. To address this issue, we suggest to use RC on imaginary data, based on fictitious facts and entities.
  arXiv  Detail & Related papers  (2024-04-09T13:08:56Z)
• Meaning and understanding in large language models [0.0]
  Recent developments in the generative large language models (LLMs) of artificial intelligence have led to the belief that traditional philosophical assumptions about machine understanding of language need to be revised. This article critically evaluates the prevailing tendency to regard machine language performance as mere syntactic manipulation and the simulation of understanding, which is only partial and very shallow, without sufficient referential grounding in the world.
  arXiv  Detail & Related papers  (2023-10-26T14:06:14Z)
• Stochastic LLMs do not Understand Language: Towards Symbolic, Explainable and Ontologically Based LLMs [0.0]
  We argue that the relative success of data-driven large language models (LLMs) is not a reflection on the symbolic vs. subsymbolic debate. We suggest in this paper applying the effective bottom-up strategy in a symbolic setting resulting in symbolic, explainable, and ontologically grounded language models.
  arXiv  Detail & Related papers  (2023-09-12T02:14:05Z)
• Symbolic and Language Agnostic Large Language Models [0.0]
  We argue that the relative success of large language models (LLMs) is not a reflection on the symbolic vs. subsymbolic debate. We suggest here is employing the successful bottom-up strategy in a symbolic setting, producing symbolic, language agnostic and ontologically grounded large language models.
  arXiv  Detail & Related papers  (2023-08-27T20:24:33Z)
• Language models are not naysayers: An analysis of language models on negation benchmarks [58.32362243122714]
  We evaluate the ability of current-generation auto-regressive language models to handle negation. We show that LLMs have several limitations including insensitivity to the presence of negation, an inability to capture the lexical semantics of negation, and a failure to reason under negation.
  arXiv  Detail & Related papers  (2023-06-14T01:16:37Z)
• Towards Explainable and Language-Agnostic LLMs: Symbolic Reverse Engineering of Language at Scale [0.0]
  Large language models (LLMs) have achieved a milestone that undenia-bly changed many held beliefs in artificial intelligence (AI) We argue for a bottom-up reverse engineering of language in a symbolic setting.
  arXiv  Detail & Related papers  (2023-05-30T15:15:40Z)
• Large Language Models are In-Context Semantic Reasoners rather than Symbolic Reasoners [75.85554779782048]
  Large Language Models (LLMs) have excited the natural language and machine learning community over recent years. Despite of numerous successful applications, the underlying mechanism of such in-context capabilities still remains unclear. In this work, we hypothesize that the learned textitsemantics of language tokens do the most heavy lifting during the reasoning process.
  arXiv  Detail & Related papers  (2023-05-24T07:33:34Z)
• Transparency Helps Reveal When Language Models Learn Meaning [71.96920839263457]
  Our systematic experiments with synthetic data reveal that, with languages where all expressions have context-independent denotations, both autoregressive and masked language models learn to emulate semantic relations between expressions. Turning to natural language, our experiments with a specific phenomenon -- referential opacity -- add to the growing body of evidence that current language models do not well-represent natural language semantics.
  arXiv  Detail & Related papers  (2022-10-14T02:35:19Z)
• Testing the Ability of Language Models to Interpret Figurative Language [69.59943454934799]
  Figurative and metaphorical language are commonplace in discourse. It remains an open question to what extent modern language models can interpret nonliteral phrases. We introduce Fig-QA, a Winograd-style nonliteral language understanding task.
  arXiv  Detail & Related papers  (2022-04-26T23:42:22Z)
• Provable Limitations of Acquiring Meaning from Ungrounded Form: What will Future Language Models Understand? [87.20342701232869]
  We investigate the abilities of ungrounded systems to acquire meaning. We study whether assertions enable a system to emulate representations preserving semantic relations like equivalence. We find that assertions enable semantic emulation if all expressions in the language are referentially transparent. However, if the language uses non-transparent patterns like variable binding, we show that emulation can become an uncomputable problem.
  arXiv  Detail & Related papers  (2021-04-22T01:00:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.