Multitasking Inhibits Semantic Drift

• URL: http://arxiv.org/abs/2104.07219v1
• Date: Thu, 15 Apr 2021 03:42:17 GMT
• Title: Multitasking Inhibits Semantic Drift
• Authors: Athul Paul Jacob, Mike Lewis, Jacob Andreas
• Abstract summary: We study the dynamics of learning in latent language policies (LLPs) LLPs can solve challenging long-horizon reinforcement learning problems. Previous work has found that LLP training is prone to semantic drift.
• Score: 46.71462510028727
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: When intelligent agents communicate to accomplish shared goals, how do these goals shape the agents' language? We study the dynamics of learning in latent language policies (LLPs), in which instructor agents generate natural-language subgoal descriptions and executor agents map these descriptions to low-level actions. LLPs can solve challenging long-horizon reinforcement learning problems and provide a rich model for studying task-oriented language use. But previous work has found that LLP training is prone to semantic drift (use of messages in ways inconsistent with their original natural language meanings). Here, we demonstrate theoretically and empirically that multitask training is an effective counter to this problem: we prove that multitask training eliminates semantic drift in a well-studied family of signaling games, and show that multitask training of neural LLPs in a complex strategy game reduces drift and while improving sample efficiency.

Related papers

• No Train but Gain: Language Arithmetic for training-free Language Adapters enhancement [59.37775534633868]
  We introduce a novel method called language arithmetic, which enables training-free post-processing. The effectiveness of the proposed solution is demonstrated on three downstream tasks in a MAD-X-based set of cross-lingual schemes.
  arXiv  Detail & Related papers  (2024-04-24T08:52:40Z)
• GLIDE-RL: Grounded Language Instruction through DEmonstration in RL [7.658523833511356]
  Training efficient Reinforcement Learning (RL) agents grounded in natural language has been a long-standing challenge. We present a novel algorithm, Grounded Language Instruction through DEmonstration in RL (GLIDE-RL) that introduces a teacher-instructor-student curriculum learning framework. In this multi-agent framework, the teacher and the student agents learn simultaneously based on the student's current skill level.
  arXiv  Detail & Related papers  (2024-01-03T17:32:13Z)
• LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Models [56.25156596019168]
  This paper introduces the LMRL-Gym benchmark for evaluating multi-turn RL for large language models (LLMs) Our benchmark consists of 8 different language tasks, which require multiple rounds of language interaction and cover a range of tasks in open-ended dialogue and text games.
  arXiv  Detail & Related papers  (2023-11-30T03:59:31Z)
• Mastering Robot Manipulation with Multimodal Prompts through Pretraining and Multi-task Fine-tuning [49.92517970237088]
  We tackle the problem of training a robot to understand multimodal prompts. This type of task poses a major challenge to robots' capability to understand the interconnection and complementarity between vision and language signals. We introduce an effective framework that learns a policy to perform robot manipulation with multimodal prompts.
  arXiv  Detail & Related papers  (2023-10-14T22:24:58Z)
• Collaborating with language models for embodied reasoning [30.82976922056617]
  Reasoning in a complex and ambiguous environment is a key goal for Reinforcement Learning (RL) agents. We present a set of tasks that require reasoning, test this system's ability to generalize zero-shot and investigate failure cases.
  arXiv  Detail & Related papers  (2023-02-01T21:26:32Z)
• On the cross-lingual transferability of multilingual prototypical models across NLU tasks [2.44288434255221]
  Supervised deep learning-based approaches have been applied to task-oriented dialog and have proven to be effective for limited domain and language applications. In practice, these approaches suffer from the drawbacks of domain-driven design and under-resourced languages. This article proposes to investigate the cross-lingual transferability of using synergistically few-shot learning with prototypical neural networks and multilingual Transformers-based models.
  arXiv  Detail & Related papers  (2022-07-19T09:55:04Z)
• Context-Aware Language Modeling for Goal-Oriented Dialogue Systems [84.65707332816353]
  We formulate goal-oriented dialogue as a partially observed Markov decision process. We derive a simple and effective method to finetune language models in a goal-aware way. We evaluate our method on a practical flight-booking task using AirDialogue.
  arXiv  Detail & Related papers  (2022-04-18T17:23:11Z)
• Verb Knowledge Injection for Multilingual Event Processing [50.27826310460763]
  We investigate whether injecting explicit information on verbs' semantic-syntactic behaviour improves the performance of LM-pretrained Transformers. We first demonstrate that injecting verb knowledge leads to performance gains in English event extraction. We then explore the utility of verb adapters for event extraction in other languages.
  arXiv  Detail & Related papers  (2020-12-31T03:24:34Z)
• Ask Your Humans: Using Human Instructions to Improve Generalization in Reinforcement Learning [32.82030512053361]
  We propose the use of step-by-step human demonstrations in the form of natural language instructions and action trajectories. We find that human demonstrations help solve the most complex tasks. We also find that incorporating natural language allows the model to generalize to unseen tasks in a zero-shot setting.
  arXiv  Detail & Related papers  (2020-11-01T14:39:46Z)
• Networked Multi-Agent Reinforcement Learning with Emergent Communication [18.47483427884452]
  Multi-Agent Reinforcement Learning (MARL) methods find optimal policies for agents that operate in the presence of other learning agents. One way to coordinate is by learning to communicate with each other. Can the agents develop a language while learning to perform a common task?
  arXiv  Detail & Related papers  (2020-04-06T16:13:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.