Learning and communication pressures in neural networks: Lessons from emergent communication

• URL: http://arxiv.org/abs/2403.14427v3
• Date: Wed, 27 Nov 2024 14:58:17 GMT
• Title: Learning and communication pressures in neural networks: Lessons from emergent communication
• Authors: Lukas Galke, Limor Raviv,
• Abstract summary: We look at three cases where mismatches between the emergent linguistic behavior of neural agents and humans were resolved. We identify key pressures at play for language learning and emergence: communicative success, production effort, learnability, and other psycho-/sociolinguistic factors.
• Score: 5.371337604556311
• License:
• Abstract: Finding and facilitating commonalities between the linguistic behaviors of large language models and humans could lead to major breakthroughs in our understanding of the acquisition, processing, and evolution of language. However, most findings on human-LLM similarity can be attributed to training on human data. The field of emergent machine-to-machine communication provides an ideal testbed for discovering which pressures are neural agents naturally exposed to when learning to communicate in isolation, without any human language to start with. Here, we review three cases where mismatches between the emergent linguistic behavior of neural agents and humans were resolved thanks to introducing theoretically-motivated inductive biases. By contrasting humans, large language models, and emergent communication agents, we then identify key pressures at play for language learning and emergence: communicative success, production effort, learnability, and other psycho-/sociolinguistic factors. We discuss their implications and relevance to the field of language evolution and acquisition. By mapping out the necessary inductive biases that make agents' emergent languages more human-like, we not only shed light on the underlying principles of human cognition and communication, but also inform and improve the very use of these models as valuable scientific tools for studying language learning, processing, use, and representation more broadly.

Related papers

• Developmental Predictive Coding Model for Early Infancy Mono and Bilingual Vocal Continual Learning [69.8008228833895]
  We propose a small-sized generative neural network equipped with a continual learning mechanism. Our model prioritizes interpretability and demonstrates the advantages of online learning.
  arXiv  Detail & Related papers  (2024-12-23T10:23:47Z)
• Towards More Human-like AI Communication: A Review of Emergent Communication Research [0.0]
  Emergent communication (Emecom) is a field of research aiming to develop artificial agents capable of using natural language. In this review, we delineate all the common proprieties we find across the literature and how they relate to human interactions. We identify two subcategories and highlight their characteristics and open challenges.
  arXiv  Detail & Related papers  (2023-08-01T14:43:10Z)
• Communication Drives the Emergence of Language Universals in Neural Agents: Evidence from the Word-order/Case-marking Trade-off [3.631024220680066]
  We propose a new Neural-agent Language Learning and Communication framework (NeLLCom) where pairs of speaking and listening agents first learn a miniature language. We succeed in replicating the trade-off with the new framework without hard-coding specific biases in the agents.
  arXiv  Detail & Related papers  (2023-01-30T17:22:33Z)
• What Artificial Neural Networks Can Tell Us About Human Language Acquisition [47.761188531404066]
  Rapid progress in machine learning for natural language processing has the potential to transform debates about how humans learn language. To increase the relevance of learnability results from computational models, we need to train model learners without significant advantages over humans.
  arXiv  Detail & Related papers  (2022-08-17T00:12:37Z)
• Emergent Communication for Understanding Human Language Evolution: What's Missing? [1.2891210250935146]
  We discuss three important phenomena with respect to the emergence and benefits of compositionality. We argue that one possible reason for these mismatches is that key cognitive and communicative constraints of humans are not yet integrated.
  arXiv  Detail & Related papers  (2022-04-22T09:21:53Z)
• Few-shot Language Coordination by Modeling Theory of Mind [95.54446989205117]
  We study the task of few-shot $textitlanguage coordination$. We require the lead agent to coordinate with a $textitpopulation$ of agents with different linguistic abilities. This requires the ability to model the partner's beliefs, a vital component of human communication.
  arXiv  Detail & Related papers  (2021-07-12T19:26:11Z)
• Cetacean Translation Initiative: a roadmap to deciphering the communication of sperm whales [97.41394631426678]
  Recent research showed the promise of machine learning tools for analyzing acoustic communication in nonhuman species. We outline the key elements required for the collection and processing of massive bioacoustic data of sperm whales. The technological capabilities developed are likely to yield cross-applications and advancements in broader communities investigating non-human communication and animal behavioral research.
  arXiv  Detail & Related papers  (2021-04-17T18:39:22Z)
• Emergent Multi-Agent Communication in the Deep Learning Era [26.764052787245728]
  The ability to cooperate through language is a defining feature of humans. As the perceptual, motory and planning capabilities of deep artificial networks increase, researchers are studying whether they also can develop a shared language to interact.
  arXiv  Detail & Related papers  (2020-06-03T17:50:16Z)
• Experience Grounds Language [185.73483760454454]
  Language understanding research is held back by a failure to relate language to the physical world it describes and to the social interactions it facilitates. Despite the incredible effectiveness of language processing models to tackle tasks after being trained on text alone, successful linguistic communication relies on a shared experience of the world.
  arXiv  Detail & Related papers  (2020-04-21T16:56:27Z)
• On the interaction between supervision and self-play in emergent communication [82.290338507106]
  We investigate the relationship between two categories of learning signals with the ultimate goal of improving sample efficiency. We find that first training agents via supervised learning on human data followed by self-play outperforms the converse.
  arXiv  Detail & Related papers  (2020-02-04T02:35:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.