Towards Graph Representation Learning in Emergent Communication

• URL: http://arxiv.org/abs/2001.09063v2
• Date: Tue, 4 Feb 2020 14:18:31 GMT
• Title: Towards Graph Representation Learning in Emergent Communication
• Authors: Agnieszka S{\l}owik, Abhinav Gupta, William L. Hamilton, Mateja Jamnik, Sean B. Holden
• Abstract summary: We use graph convolutional networks to support the evolution of language and cooperation in multi-agent systems. Motivated by an image-based referential game, we propose a graph referential game with varying degrees of complexity. We show that the emerged communication protocol is robust, that the agents uncover the true factors of variation in the game, and that they learn to generalize beyond the samples encountered during training.
• Score: 37.8523331078468
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent findings in neuroscience suggest that the human brain represents information in a geometric structure (for instance, through conceptual spaces). In order to communicate, we flatten the complex representation of entities and their attributes into a single word or a sentence. In this paper we use graph convolutional networks to support the evolution of language and cooperation in multi-agent systems. Motivated by an image-based referential game, we propose a graph referential game with varying degrees of complexity, and we provide strong baseline models that exhibit desirable properties in terms of language emergence and cooperation. We show that the emerged communication protocol is robust, that the agents uncover the true factors of variation in the game, and that they learn to generalize beyond the samples encountered during training.

Related papers

• Coarse-to-Fine Contrastive Learning in Image-Text-Graph Space for Improved Vision-Language Compositionality [50.48859793121308]
  Contrastively trained vision-language models have achieved remarkable progress in vision and language representation learning. Recent research has highlighted severe limitations in their ability to perform compositional reasoning over objects, attributes, and relations.
  arXiv  Detail & Related papers  (2023-05-23T08:28:38Z)
• A framework for the emergence and analysis of language in social learning agents [0.0]
  This study proposes a communication protocol between cooperative agents to analyze the formation of individual and shared abstractions. Using grid-world mazes and reinforcement learning, teacher ANNs pass a compressed message to a student ANN for better task completion. This highlights the role of language as a common representation between agents and its implications on generalization capabilities.
  arXiv  Detail & Related papers  (2023-05-04T08:11:01Z)
• Emergence of Shared Sensory-motor Graphical Language from Visual Input [22.23299485364174]
  We introduce the Graphical Referential Game (GREG) where a speaker must produce a graphical utterance to name a visual referent object. The utterances are drawing images produced using dynamical motor primitives combined with a sketching library. We show that our method allows the emergence of a shared, graphical language with compositional properties.
  arXiv  Detail & Related papers  (2022-10-03T17:11:18Z)
• Compositional Generalization in Grounded Language Learning via Induced Model Sparsity [81.38804205212425]
  We consider simple language-conditioned navigation problems in a grid world environment with disentangled observations. We design an agent that encourages sparse correlations between words in the instruction and attributes of objects, composing them together to find the goal. Our agent maintains a high level of performance on goals containing novel combinations of properties even when learning from a handful of demonstrations.
  arXiv  Detail & Related papers  (2022-07-06T08:46:27Z)
• Oracle Linguistic Graphs Complement a Pretrained Transformer Language Model: A Cross-formalism Comparison [13.31232311913236]
  We examine the extent to which, in principle, linguistic graph representations can complement and improve neural language modeling. We find that, overall, semantic constituency structures are most useful to language modeling performance.
  arXiv  Detail & Related papers  (2021-12-15T04:29:02Z)
• Emergent Graphical Conventions in a Visual Communication Game [80.79297387339614]
  Humans communicate with graphical sketches apart from symbolic languages. We take the very first step to model and simulate such an evolution process via two neural agents playing a visual communication game. We devise a novel reinforcement learning method such that agents are evolved jointly towards successful communication and abstract graphical conventions.
  arXiv  Detail & Related papers  (2021-11-28T18:59:57Z)
• Prototypical Representation Learning for Relation Extraction [56.501332067073065]
  This paper aims to learn predictive, interpretable, and robust relation representations from distantly-labeled data. We learn prototypes for each relation from contextual information to best explore the intrinsic semantics of relations. Results on several relation learning tasks show that our model significantly outperforms the previous state-of-the-art relational models.
  arXiv  Detail & Related papers  (2021-03-22T08:11:43Z)
• Neuro-Symbolic Representations for Video Captioning: A Case for Leveraging Inductive Biases for Vision and Language [148.0843278195794]
  We propose a new model architecture for learning multi-modal neuro-symbolic representations for video captioning. Our approach uses a dictionary learning-based method of learning relations between videos and their paired text descriptions.
  arXiv  Detail & Related papers  (2020-11-18T20:21:19Z)
• Structural Inductive Biases in Emergent Communication [36.26083882473554]
  We investigate the impact of representation learning in artificial agents by developing graph referential games. We show that agents parametrized by graph neural networks develop a more compositional language compared to bag-of-words and sequence models.
  arXiv  Detail & Related papers  (2020-02-04T14:59:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.