Searching for the Most Human-like Emergent Language

• URL: http://arxiv.org/abs/2510.03467v1
• Date: Fri, 03 Oct 2025 19:45:01 GMT
• Title: Searching for the Most Human-like Emergent Language
• Abstract summary: We generate state-of-the-art emergent languages in terms of similarity to human language.<n>XferBench measures its suitability for deep transfer learning to human language.
• Score: 2.1700203922407497
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we design a signalling game-based emergent communication environment to generate state-of-the-art emergent languages in terms of similarity to human language. This is done with hyperparameter optimization, using XferBench as the objective function. XferBench quantifies the statistical similarity of emergent language to human language by measuring its suitability for deep transfer learning to human language. Additionally, we demonstrate the predictive power of entropy on the transfer learning performance of emergent language as well as corroborate previous results on the entropy-minimization properties of emergent communication systems. Finally, we report generalizations regarding what hyperparameters produce more realistic emergent languages, that is, ones which transfer better to human language.

Related papers

• Pretrained Multilingual Transformers Reveal Quantitative Distance Between Human Languages [25.106812852291622]
  We introduce a method that leverages pretrained multilingual language models as systematic instruments for linguistic measurement.<n>We show that the spontaneously emerged attention mechanisms of these models provide a robust, tokenization-agnostic measure of cross-linguistic distance.<n>Our results establish a principled foundation for testing linguistic hypotheses using artificial neural networks.
  arXiv  Detail & Related papers  (2026-03-18T16:50:23Z)
• Entropy2Vec: Crosslingual Language Modeling Entropy as End-to-End Learnable Language Representations [33.52308723119687]
  We introduce Entropy2Vec, a framework for deriving cross-lingual language representations by leveraging the entropy of monolingual language models.<n>By training a language model on a single language, we hypothesize that the entropy of its predictions reflects its structural similarity to other languages.<n>This approach yields dense, non-sparse language embeddings that are adaptable to different timeframes and free from missing values.
  arXiv  Detail & Related papers  (2025-09-05T12:40:31Z)
• Investigating and Scaling up Code-Switching for Multilingual Language Model Pre-Training [58.696660064190475]
  We find that the existence of code-switching, alternating between different languages within a context, is key to multilingual capabilities.<n>To better explore the power of code-switching for language alignment during pre-training, we investigate the strategy of synthetic code-switching.
  arXiv  Detail & Related papers  (2025-04-02T15:09:58Z)
• Searching for Structure: Investigating Emergent Communication with Large Language Models [0.10923877073891446]
  We simulate a classical referential game in which Large Language Models learn and use artificial languages.<n>Our results show that initially unstructured holistic languages are indeed shaped to have some structural properties that allow two LLM agents to communicate successfully.
  arXiv  Detail & Related papers  (2024-12-10T16:32:19Z)
• Analyzing The Language of Visual Tokens [48.62180485759458]
  We take a natural-language-centric approach to analyzing discrete visual languages. We show that higher token innovation drives greater entropy and lower compression, with tokens predominantly representing object parts. We also show that visual languages lack cohesive grammatical structures, leading to higher perplexity and weaker hierarchical organization compared to natural languages.
  arXiv  Detail & Related papers  (2024-11-07T18:59:28Z)
• XferBench: a Data-Driven Benchmark for Emergent Language [1.6574413179773761]
  We introduce a benchmark for evaluating the overall quality of emergent languages. We measure this by using the emergent language as pretraining data for a downstream NLP tasks in human language. We empirically test the benchmark's validity using human, synthetic, and emergent language baselines.
  arXiv  Detail & Related papers  (2024-07-03T19:02:26Z)
• Learning and communication pressures in neural networks: Lessons from emergent communication [5.371337604556311]
  We look at three cases where mismatches between the emergent linguistic behavior of neural agents and humans were resolved.<n>We identify key pressures at play for language learning and emergence: communicative success, production effort, learnability, and other psycho-/sociolinguistic factors.
  arXiv  Detail & Related papers  (2024-03-21T14:33:34Z)
• Soft Language Clustering for Multilingual Model Pre-training [57.18058739931463]
  We propose XLM-P, which contextually retrieves prompts as flexible guidance for encoding instances conditionally. Our XLM-P enables (1) lightweight modeling of language-invariant and language-specific knowledge across languages, and (2) easy integration with other multilingual pre-training methods.
  arXiv  Detail & Related papers  (2023-06-13T08:08:08Z)
• Linking Emergent and Natural Languages via Corpus Transfer [98.98724497178247]
  We propose a novel way to establish a link by corpus transfer between emergent languages and natural languages. Our approach showcases non-trivial transfer benefits for two different tasks -- language modeling and image captioning. We also introduce a novel metric to predict the transferability of an emergent language by translating emergent messages to natural language captions grounded on the same images.
  arXiv  Detail & Related papers  (2022-03-24T21:24:54Z)
• Towards Zero-shot Language Modeling [90.80124496312274]
  We construct a neural model that is inductively biased towards learning human languages. We infer this distribution from a sample of typologically diverse training languages. We harness additional language-specific side information as distant supervision for held-out languages.
  arXiv  Detail & Related papers  (2021-08-06T23:49:18Z)
• Emergent Communication Pretraining for Few-Shot Machine Translation [66.48990742411033]
  We pretrain neural networks via emergent communication from referential games. Our key assumption is that grounding communication on images---as a crude approximation of real-world environments---inductively biases the model towards learning natural languages.
  arXiv  Detail & Related papers  (2020-11-02T10:57:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.