From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thought

• URL: http://arxiv.org/abs/2306.12672v2
• Date: Fri, 23 Jun 2023 06:05:31 GMT
• Title: From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thought
• Authors: Lionel Wong, Gabriel Grand, Alexander K. Lew, Noah D. Goodman, Vikash K. Mansinghka, Jacob Andreas, Joshua B. Tenenbaum
• Abstract summary: We propose rational meaning construction, a computational framework for language-informed thinking. We frame linguistic meaning as a context-sensitive mapping from natural language into a probabilistic language of thought. We show that LLMs can generate context-sensitive translations that capture pragmatically-appropriate linguistic meanings. We extend our framework to integrate cognitively-motivated symbolic modules.
• Score: 124.40905824051079
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: How does language inform our downstream thinking? In particular, how do humans make meaning from language--and how can we leverage a theory of linguistic meaning to build machines that think in more human-like ways? In this paper, we propose rational meaning construction, a computational framework for language-informed thinking that combines neural language models with probabilistic models for rational inference. We frame linguistic meaning as a context-sensitive mapping from natural language into a probabilistic language of thought (PLoT)--a general-purpose symbolic substrate for generative world modeling. Our architecture integrates two computational tools that have not previously come together: we model thinking with probabilistic programs, an expressive representation for commonsense reasoning; and we model meaning construction with large language models (LLMs), which support broad-coverage translation from natural language utterances to code expressions in a probabilistic programming language. We illustrate our framework through examples covering four core domains from cognitive science: probabilistic reasoning, logical and relational reasoning, visual and physical reasoning, and social reasoning. In each, we show that LLMs can generate context-sensitive translations that capture pragmatically-appropriate linguistic meanings, while Bayesian inference with the generated programs supports coherent and robust commonsense reasoning. We extend our framework to integrate cognitively-motivated symbolic modules (physics simulators, graphics engines, and planning algorithms) to provide a unified commonsense thinking interface from language. Finally, we explore how language can drive the construction of world models themselves. We hope this work will provide a roadmap towards cognitive models and AI systems that synthesize the insights of both modern and classical computational perspectives.

Related papers

• Proceedings of the First International Workshop on Next-Generation Language Models for Knowledge Representation and Reasoning (NeLaMKRR 2024) [16.282850445579857]
  Reasoning is an essential component of human intelligence as it plays a fundamental role in our ability to think critically. Recent leap forward in natural language processing, with the emergence of language models based on transformers, is hinting at the possibility that these models exhibit reasoning abilities. Despite ongoing discussions about what reasoning is in language models, it is still not easy to pin down to what extent these models are actually capable of reasoning.
  arXiv  Detail & Related papers  (2024-10-07T02:31:47Z)
• Formal Aspects of Language Modeling [74.16212987886013]
  Large language models have become one of the most commonly deployed NLP inventions. These notes are the accompaniment to the theoretical portion of the ETH Z"urich course on large language models.
  arXiv  Detail & Related papers  (2023-11-07T20:21:42Z)
• Navigation with Large Language Models: Semantic Guesswork as a Heuristic for Planning [73.0990339667978]
  Navigation in unfamiliar environments presents a major challenge for robots. We use language models to bias exploration of novel real-world environments. We evaluate LFG in challenging real-world environments and simulated benchmarks.
  arXiv  Detail & Related papers  (2023-10-16T06:21:06Z)
• Large Language Models for Scientific Synthesis, Inference and Explanation [56.41963802804953]
  We show how large language models can perform scientific synthesis, inference, and explanation. We show that the large language model can augment this "knowledge" by synthesizing from the scientific literature. This approach has the further advantage that the large language model can explain the machine learning system's predictions.
  arXiv  Detail & Related papers  (2023-10-12T02:17:59Z)
• Minding Language Models' (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Tracker [72.09076317574238]
  ToM is a plug-and-play approach to investigate the belief states of characters in reading comprehension. We show that ToM enhances off-the-shelf neural network theory mind in a zero-order setting while showing robust out-of-distribution performance compared to supervised baselines.
  arXiv  Detail & Related papers  (2023-06-01T17:24:35Z)
• Language Models are Bounded Pragmatic Speakers: Understanding RLHF from a Bayesian Cognitive Modeling Perspective [2.8282906214258805]
  This paper formulates a probabilistic cognitive model called the bounded pragmatic speaker. We demonstrate that large language models fine-tuned with reinforcement learning from human feedback embody a model of thought that resembles a fast-and-slow model.
  arXiv  Detail & Related papers  (2023-05-28T16:04:48Z)
• On the Computation of Meaning, Language Models and Incomprehensible Horrors [0.0]
  We integrate foundational theories of meaning with a mathematical formalism of artificial general intelligence (AGI) Our findings shed light on the relationship between meaning and intelligence, and how we can build machines that comprehend and intend meaning.
  arXiv  Detail & Related papers  (2023-04-25T09:41:00Z)
• Emergence of Machine Language: Towards Symbolic Intelligence with Neural Networks [73.94290462239061]
  We propose to combine symbolism and connectionism principles by using neural networks to derive a discrete representation. By designing an interactive environment and task, we demonstrated that machines could generate a spontaneous, flexible, and semantic language.
  arXiv  Detail & Related papers  (2022-01-14T14:54:58Z)
• Language Models are not Models of Language [0.0]
  Transfer learning has enabled large deep learning neural networks trained on the language modeling task to vastly improve performance. We argue that the term language model is misleading because deep learning models are not theoretical models of language.
  arXiv  Detail & Related papers  (2021-12-13T22:39:46Z)
• Modelling Compositionality and Structure Dependence in Natural Language [0.12183405753834563]
  Drawing on linguistics and set theory, a formalisation of these ideas is presented in the first half of this thesis. We see how cognitive systems that process language need to have certain functional constraints. Using the advances of word embedding techniques, a model of relational learning is simulated.
  arXiv  Detail & Related papers  (2020-11-22T17:28:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.