Visual Theory of Mind Enables the Invention of Writing Systems

• URL: http://arxiv.org/abs/2502.01568v2
• Date: Tue, 04 Feb 2025 03:36:30 GMT
• Title: Visual Theory of Mind Enables the Invention of Writing Systems
• Authors: Benjamin A. Spiegel, Lucas Gelfond, George Konidaris,
• Abstract summary: Evidence suggests that the earliest forms of some writing systems originally consisted of iconic pictographs. Our model sheds light on the cognitive and cultural processes that led to the development of early writing systems.
• Score: 10.013537728631038
• License:
• Abstract: Abstract symbolic writing systems are semiotic codes that are ubiquitous in modern society but are otherwise absent in the animal kingdom. Anthropological evidence suggests that the earliest forms of some writing systems originally consisted of iconic pictographs, which signify their referent via visual resemblance. While previous studies have examined the emergence and, separately, the evolution of pictographic writing systems through a computational lens, most employ non-naturalistic methodologies that make it difficult to draw clear analogies to human and animal cognition. We develop a multi-agent reinforcement learning testbed for emergent communication called a Signification Game, and formulate a model of inferential communication that enables agents to leverage visual theory of mind to communicate actions using pictographs. Our model, which is situated within a broader formalism for animal communication, sheds light on the cognitive and cultural processes that led to the development of early writing systems.

Related papers

• Science is Exploration: Computational Frontiers for Conceptual Metaphor Theory [0.0]
  We show that Large Language Models (LLMs) can accurately identify and explain the presence of conceptual metaphors in natural language data. Using a novel prompting technique based on metaphor annotation guidelines, we demonstrate that LLMs are a promising tool for large-scale computational research on conceptual metaphors.
  arXiv  Detail & Related papers  (2024-10-11T17:03:13Z)
• The Language of Infographics: Toward Understanding Conceptual Metaphor Use in Scientific Storytelling [9.302187675469554]
  We map Conceptual Metaphor (CMT) to the visualization domain to address patterns of visual conceptual metaphors that are often used in science infographics. Our analysis shows that ontological and orientational conceptual metaphors are the most widely applied to translate complex scientific concepts.
  arXiv  Detail & Related papers  (2024-07-18T11:39:50Z)
• Models of symbol emergence in communication: a conceptual review and a guide for avoiding local minima [0.0]
  Computational simulations are a popular method for testing hypotheses about the emergence of communication. We identify the assumptions and explanatory targets of several most representative models and summarise the known results. In line with this perspective, we sketch the road towards modelling the emergence of meaningful symbolic communication.
  arXiv  Detail & Related papers  (2023-03-08T12:53:03Z)
• CLAMP: Prompt-based Contrastive Learning for Connecting Language and Animal Pose [70.59906971581192]
  We introduce a novel prompt-based Contrastive learning scheme for connecting Language and AniMal Pose effectively. The CLAMP attempts to bridge the gap by adapting the text prompts to the animal keypoints during network training. Experimental results show that our method achieves state-of-the-art performance under the supervised, few-shot, and zero-shot settings.
  arXiv  Detail & Related papers  (2022-06-23T14:51:42Z)
• Emergent Graphical Conventions in a Visual Communication Game [80.79297387339614]
  Humans communicate with graphical sketches apart from symbolic languages. We take the very first step to model and simulate such an evolution process via two neural agents playing a visual communication game. We devise a novel reinforcement learning method such that agents are evolved jointly towards successful communication and abstract graphical conventions.
  arXiv  Detail & Related papers  (2021-11-28T18:59:57Z)
• pix2rule: End-to-end Neuro-symbolic Rule Learning [84.76439511271711]
  This paper presents a complete neuro-symbolic method for processing images into objects, learning relations and logical rules. The main contribution is a differentiable layer in a deep learning architecture from which symbolic relations and rules can be extracted. We demonstrate that our model scales beyond state-of-the-art symbolic learners and outperforms deep relational neural network architectures.
  arXiv  Detail & Related papers  (2021-06-14T15:19:06Z)
• Metaphor Generation with Conceptual Mappings [58.61307123799594]
  We aim to generate a metaphoric sentence given a literal expression by replacing relevant verbs. We propose to control the generation process by encoding conceptual mappings between cognitive domains. We show that the unsupervised CM-Lex model is competitive with recent deep learning metaphor generation systems.
  arXiv  Detail & Related papers  (2021-06-02T15:27:05Z)
• Cetacean Translation Initiative: a roadmap to deciphering the communication of sperm whales [97.41394631426678]
  Recent research showed the promise of machine learning tools for analyzing acoustic communication in nonhuman species. We outline the key elements required for the collection and processing of massive bioacoustic data of sperm whales. The technological capabilities developed are likely to yield cross-applications and advancements in broader communities investigating non-human communication and animal behavioral research.
  arXiv  Detail & Related papers  (2021-04-17T18:39:22Z)
• Natural Language Rationales with Full-Stack Visual Reasoning: From Pixels to Semantic Frames to Commonsense Graphs [106.15931418425906]
  We present the first study focused on generating natural language rationales across several complex visual reasoning tasks. We present RationaleVT Transformer, an integrated model that learns to generate free-text rationales by combining pretrained language models with object recognition, grounded visual semantic frames, and visual commonsense graphs. Our experiments show that the base pretrained language model benefits from visual adaptation and that free-text rationalization is a promising research direction to complement model interpretability for complex visual-textual reasoning tasks.
  arXiv  Detail & Related papers  (2020-10-15T05:08:56Z)
• Towards Graph Representation Learning in Emergent Communication [37.8523331078468]
  We use graph convolutional networks to support the evolution of language and cooperation in multi-agent systems. Motivated by an image-based referential game, we propose a graph referential game with varying degrees of complexity. We show that the emerged communication protocol is robust, that the agents uncover the true factors of variation in the game, and that they learn to generalize beyond the samples encountered during training.
  arXiv  Detail & Related papers  (2020-01-24T15:55:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.