Internal and external pressures on language emergence: least effort, object constancy and frequency

• URL: http://arxiv.org/abs/2004.03868v3
• Date: Tue, 13 Oct 2020 09:29:44 GMT
• Title: Internal and external pressures on language emergence: least effort, object constancy and frequency
• Authors: Diana Rodr\'iguez Luna, Edoardo Maria Ponti, Dieuwke Hupkes, Elia Bruni
• Abstract summary: In previous work, artificial agents were shown to achieve almost perfect accuracy in referential games where they have to communicate to identify images. We propose some realistic sources of pressure on communication that avert this outcome. Our findings reveal that the proposed sources of pressure result in emerging languages with less redundancy, more focus on high-level conceptual information, and better abilities of generalisation.
• Score: 27.731900533634516
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In previous work, artificial agents were shown to achieve almost perfect accuracy in referential games where they have to communicate to identify images. Nevertheless, the resulting communication protocols rarely display salient features of natural languages, such as compositionality. In this paper, we propose some realistic sources of pressure on communication that avert this outcome. More specifically, we formalise the principle of least effort through an auxiliary objective. Moreover, we explore several game variants, inspired by the principle of object constancy, in which we alter the frequency, position, and luminosity of the objects in the images. We perform an extensive analysis on their effect through compositionality metrics, diagnostic classifiers, and zero-shot evaluation. Our findings reveal that the proposed sources of pressure result in emerging languages with less redundancy, more focus on high-level conceptual information, and better abilities of generalisation. Overall, our contributions reduce the gap between emergent and natural languages.

Related papers

• Analyzing The Language of Visual Tokens [48.62180485759458]
  We take a natural-language-centric approach to analyzing discrete visual languages. We show that higher token innovation drives greater entropy and lower compression, with tokens predominantly representing object parts. We also show that visual languages lack cohesive grammatical structures, leading to higher perplexity and weaker hierarchical organization compared to natural languages.
  arXiv  Detail & Related papers  (2024-11-07T18:59:28Z)
• The Curious Case of Representational Alignment: Unravelling Visio-Linguistic Tasks in Emergent Communication [1.3499500088995464]
  We assess the representational alignment between agent image representations and agent representations and input images. We identify a strong relationship between inter-agent alignment and topographic similarity, a common metric for compositionality. Our findings emphasise the key role representational alignment plays in simulations of language emergence.
  arXiv  Detail & Related papers  (2024-07-25T11:29:27Z)
• On the Robustness of Language Guidance for Low-Level Vision Tasks: Findings from Depth Estimation [71.72465617754553]
  We generate "low-level" sentences that convey object-centric, three-dimensional spatial relationships, incorporate them as additional language priors and evaluate their downstream impact on depth estimation. Our key finding is that current language-guided depth estimators perform optimally only with scene-level descriptions. Despite leveraging additional data, these methods are not robust to directed adversarial attacks and decline in performance with an increase in distribution shift.
  arXiv  Detail & Related papers  (2024-04-12T15:35:20Z)
• Learning Multi-Object Positional Relationships via Emergent Communication [16.26264889682904]
  We train agents in a referential game where observations contain two objects, and find that generalization is the major problem when the positional relationship is involved. We find that the learned language can generalize well in a new multi-step MDP task where the positional relationship describes the goal, and performs better than raw-pixel images as well as pre-trained image features. We also show that language transfer from the referential game performs better in the new task than learning language directly in this task, implying the potential benefits of pre-training in referential games.
  arXiv  Detail & Related papers  (2023-02-16T04:44:53Z)
• Compositional Generalization in Grounded Language Learning via Induced Model Sparsity [81.38804205212425]
  We consider simple language-conditioned navigation problems in a grid world environment with disentangled observations. We design an agent that encourages sparse correlations between words in the instruction and attributes of objects, composing them together to find the goal. Our agent maintains a high level of performance on goals containing novel combinations of properties even when learning from a handful of demonstrations.
  arXiv  Detail & Related papers  (2022-07-06T08:46:27Z)
• Linking Emergent and Natural Languages via Corpus Transfer [98.98724497178247]
  We propose a novel way to establish a link by corpus transfer between emergent languages and natural languages. Our approach showcases non-trivial transfer benefits for two different tasks -- language modeling and image captioning. We also introduce a novel metric to predict the transferability of an emergent language by translating emergent messages to natural language captions grounded on the same images.
  arXiv  Detail & Related papers  (2022-03-24T21:24:54Z)
• A Latent-Variable Model for Intrinsic Probing [93.62808331764072]
  We propose a novel latent-variable formulation for constructing intrinsic probes. We find empirical evidence that pre-trained representations develop a cross-lingually entangled notion of morphosyntax.
  arXiv  Detail & Related papers  (2022-01-20T15:01:12Z)
• Towards Zero-shot Language Modeling [90.80124496312274]
  We construct a neural model that is inductively biased towards learning human languages. We infer this distribution from a sample of typologically diverse training languages. We harness additional language-specific side information as distant supervision for held-out languages.
  arXiv  Detail & Related papers  (2021-08-06T23:49:18Z)
• Emergent Communication of Generalizations [13.14792537601313]
  We argue that communicating about a single object in a shared visual context is prone to overfitting and does not encourage language useful beyond concrete reference. We propose games that require communicating generalizations over sets of objects representing abstract visual concepts. We find that these games greatly improve systematicity and interpretability of the learned languages.
  arXiv  Detail & Related papers  (2021-06-04T19:02:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.