Emergent Language Generalization and Acquisition Speed are not tied to Compositionality

• URL: http://arxiv.org/abs/2004.03420v2
• Date: Sat, 25 Apr 2020 14:46:24 GMT
• Title: Emergent Language Generalization and Acquisition Speed are not tied to Compositionality
• Authors: Eugene Kharitonov and Marco Baroni
• Abstract summary: We argue that these beneficial properties are only loosely connected to compositionality. In two experiments, we demonstrate that, depending on the task, non-compositional languages might show equal, or better, generalization performance and acquisition speed.
• Score: 31.6793931695019
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Studies of discrete languages emerging when neural agents communicate to solve a joint task often look for evidence of compositional structure. This stems for the expectation that such a structure would allow languages to be acquired faster by the agents and enable them to generalize better. We argue that these beneficial properties are only loosely connected to compositionality. In two experiments, we demonstrate that, depending on the task, non-compositional languages might show equal, or better, generalization performance and acquisition speed than compositional ones. Further research in the area should be clearer about what benefits are expected from compositionality, and how the latter would lead to them.

Related papers

• SHINE: Saliency-aware HIerarchical NEgative Ranking for Compositional Temporal Grounding [52.98133831401225]
  Temporal grounding, also known as video moment retrieval, aims at locating video segments corresponding to a given query sentence. We propose a large language model-driven method for negative query construction, utilizing GPT-3.5-Turbo. We introduce a coarse-to-fine saliency ranking strategy, which encourages the model to learn the multi-granularity semantic relationships between videos and hierarchical negative queries.
  arXiv  Detail & Related papers  (2024-07-06T16:08:17Z)
• On Evaluating Multilingual Compositional Generalization with Translated Datasets [34.51457321680049]
  We show that compositional generalization abilities differ across languages. We craft a faithful rule-based translation of the MCWQ dataset from English to Chinese and Japanese. Even with the resulting robust benchmark, which we call MCWQ-R, we show that the distribution of compositions still suffers due to linguistic divergences.
  arXiv  Detail & Related papers  (2023-06-20T10:03:57Z)
• How Do In-Context Examples Affect Compositional Generalization? [86.57079616209474]
  In this paper, we present CoFe, a test suite to investigate in-context compositional generalization. We find that the compositional generalization performance can be easily affected by the selection of in-context examples. Our systematic experiments indicate that in-context examples should be structurally similar to the test case, diverse from each other, and individually simple.
  arXiv  Detail & Related papers  (2023-05-08T16:32:18Z)
• Are Representations Built from the Ground Up? An Empirical Examination of Local Composition in Language Models [91.3755431537592]
  Representing compositional and non-compositional phrases is critical for language understanding. We first formulate a problem of predicting the LM-internal representations of longer phrases given those of their constituents. While we would expect the predictive accuracy to correlate with human judgments of semantic compositionality, we find this is largely not the case.
  arXiv  Detail & Related papers  (2022-10-07T14:21:30Z)
• Defending Compositionality in Emergent Languages [3.04585143845864]
  We argue that some conclusions are too strong and/or incomplete. In the context of a two-agent communication game, we show that compositionality indeed seems essential for successful generalization when the evaluation is done on a proper dataset.
  arXiv  Detail & Related papers  (2022-06-09T20:13:46Z)
• Compositional Generalization Requires Compositional Parsers [69.77216620997305]
  We compare sequence-to-sequence models and models guided by compositional principles on the recent COGS corpus. We show structural generalization is a key measure of compositional generalization and requires models that are aware of complex structure.
  arXiv  Detail & Related papers  (2022-02-24T07:36:35Z)
• The paradox of the compositionality of natural language: a neural machine translation case study [15.37696298313134]
  We re-instantiate three compositionality tests from the literature and reformulate them for neural machine translation (NMT) The results highlight two main issues: the inconsistent behaviour of NMT models and their inability to (correctly) modulate between local and global processing.
  arXiv  Detail & Related papers  (2021-08-12T17:57:23Z)
• Compositional Processing Emerges in Neural Networks Solving Math Problems [100.80518350845668]
  Recent progress in artificial neural networks has shown that when large models are trained on enough linguistic data, grammatical structure emerges in their representations. We extend this work to the domain of mathematical reasoning, where it is possible to formulate precise hypotheses about how meanings should be composed. Our work shows that neural networks are not only able to infer something about the structured relationships implicit in their training data, but can also deploy this knowledge to guide the composition of individual meanings into composite wholes.
  arXiv  Detail & Related papers  (2021-05-19T07:24:42Z)
• On (Emergent) Systematic Generalisation and Compositionality in Visual Referential Games with Straight-Through Gumbel-Softmax Estimator [0.30458514384586394]
  The drivers of compositionality emerge when two (or more) agents play a non-visual referential game. This paper investigates what extent the drivers of compositionality identified so far in the field apply in the ST-GS context. Using the ST-GS approach with small batch sizes and an overcomplete communication channel improves compositionality in the emerging languages.
  arXiv  Detail & Related papers  (2020-12-19T20:40:09Z)
• Compositionality and Generalization in Emergent Languages [42.68870559695238]
  We study whether the language emerging in deep multi-agent simulations possesses a similar ability to refer to novel primitive combinations. We find no correlation between the degree of compositionality of an emergent language and its ability to generalize. The more compositional a language is, the more easily it will be picked up by new learners.
  arXiv  Detail & Related papers  (2020-04-20T08:30:14Z)
• Probing Linguistic Features of Sentence-Level Representations in Neural Relation Extraction [80.38130122127882]
  We introduce 14 probing tasks targeting linguistic properties relevant to neural relation extraction (RE) We use them to study representations learned by more than 40 different encoder architecture and linguistic feature combinations trained on two datasets. We find that the bias induced by the architecture and the inclusion of linguistic features are clearly expressed in the probing task performance.
  arXiv  Detail & Related papers  (2020-04-17T09:17:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.