Teaching Embodied Reinforcement Learning Agents: Informativeness and Diversity of Language Use

• URL: http://arxiv.org/abs/2410.24218v1
• Date: Thu, 31 Oct 2024 17:59:52 GMT
• Title: Teaching Embodied Reinforcement Learning Agents: Informativeness and Diversity of Language Use
• Authors: Jiajun Xi, Yinong He, Jianing Yang, Yinpei Dai, Joyce Chai,
• Abstract summary: It is desirable for embodied agents to have the ability to leverage human language to gain explicit or implicit knowledge for learning tasks. It's not clear how to incorporate rich language use to facilitate task learning. This paper studies different types of language inputs in facilitating reinforcement learning.
• Score: 16.425032085699698
• License:
• Abstract: In real-world scenarios, it is desirable for embodied agents to have the ability to leverage human language to gain explicit or implicit knowledge for learning tasks. Despite recent progress, most previous approaches adopt simple low-level instructions as language inputs, which may not reflect natural human communication. It's not clear how to incorporate rich language use to facilitate task learning. To address this question, this paper studies different types of language inputs in facilitating reinforcement learning (RL) embodied agents. More specifically, we examine how different levels of language informativeness (i.e., feedback on past behaviors and future guidance) and diversity (i.e., variation of language expressions) impact agent learning and inference. Our empirical results based on four RL benchmarks demonstrate that agents trained with diverse and informative language feedback can achieve enhanced generalization and fast adaptation to new tasks. These findings highlight the pivotal role of language use in teaching embodied agents new tasks in an open world. Project website: https://github.com/sled-group/Teachable_RL

Related papers

• Language Guided Skill Discovery [56.84356022198222]
  We introduce Language Guided Skill Discovery (LGSD) to maximize semantic diversity between skills. LGSD takes user prompts as input and outputs a set of semantically distinctive skills. We demonstrate that LGSD enables legged robots to visit different user-intended areas on a plane by simply changing the prompt.
  arXiv  Detail & Related papers  (2024-06-07T04:25:38Z)
• Learning to Model the World with Language [100.76069091703505]
  To interact with humans and act in the world, agents need to understand the range of language that people use and relate it to the visual world. Our key idea is that agents should interpret such diverse language as a signal that helps them predict the future. We instantiate this in Dynalang, an agent that learns a multimodal world model to predict future text and image representations.
  arXiv  Detail & Related papers  (2023-07-31T17:57:49Z)
• Simple Embodied Language Learning as a Byproduct of Meta-Reinforcement Learning [56.07190845063208]
  We ask: can embodied reinforcement learning (RL) agents indirectly learn language from non-language tasks? We design an office navigation environment, where the agent's goal is to find a particular office, and office locations differ in different buildings (i.e., tasks) We find RL agents indeed are able to indirectly learn language. Agents trained with current meta-RL algorithms successfully generalize to reading floor plans with held-out layouts and language phrases.
  arXiv  Detail & Related papers  (2023-06-14T09:48:48Z)
• Transforming Human-Centered AI Collaboration: Redefining Embodied Agents Capabilities through Interactive Grounded Language Instructions [23.318236094953072]
  Human intelligence's adaptability is remarkable, allowing us to adjust to new tasks and multi-modal environments swiftly. The research community is actively pursuing the development of interactive "embodied agents" These agents must possess the ability to promptly request feedback in case communication breaks down or instructions are unclear.
  arXiv  Detail & Related papers  (2023-05-18T07:51:33Z)
• Communication Drives the Emergence of Language Universals in Neural Agents: Evidence from the Word-order/Case-marking Trade-off [3.631024220680066]
  We propose a new Neural-agent Language Learning and Communication framework (NeLLCom) where pairs of speaking and listening agents first learn a miniature language. We succeed in replicating the trade-off with the new framework without hard-coding specific biases in the agents.
  arXiv  Detail & Related papers  (2023-01-30T17:22:33Z)
• On the cross-lingual transferability of multilingual prototypical models across NLU tasks [2.44288434255221]
  Supervised deep learning-based approaches have been applied to task-oriented dialog and have proven to be effective for limited domain and language applications. In practice, these approaches suffer from the drawbacks of domain-driven design and under-resourced languages. This article proposes to investigate the cross-lingual transferability of using synergistically few-shot learning with prototypical neural networks and multilingual Transformers-based models.
  arXiv  Detail & Related papers  (2022-07-19T09:55:04Z)
• Cross-lingual Lifelong Learning [53.06904052325966]
  We present a principled Cross-lingual Continual Learning (CCL) evaluation paradigm. We provide insights into what makes multilingual sequential learning particularly challenging. The implications of this analysis include a recipe for how to measure and balance different cross-lingual continual learning desiderata.
  arXiv  Detail & Related papers  (2022-05-23T09:25:43Z)
• LISA: Learning Interpretable Skill Abstractions from Language [85.20587800593293]
  We propose a hierarchical imitation learning framework that can learn diverse, interpretable skills from language-conditioned demonstrations. Our method demonstrates a more natural way to condition on language in sequential decision-making problems.
  arXiv  Detail & Related papers  (2022-02-28T19:43:24Z)
• Information-Theoretic Probing for Linguistic Structure [74.04862204427944]
  We propose an information-theoretic operationalization of probing as estimating mutual information. We evaluate on a set of ten typologically diverse languages often underrepresented in NLP research.
  arXiv  Detail & Related papers  (2020-04-07T01:06:36Z)
• Zero-Shot Cross-Lingual Transfer with Meta Learning [45.29398184889296]
  We consider the setting of training models on multiple languages at the same time, when little or no data is available for languages other than English. We show that this challenging setup can be approached using meta-learning. We experiment using standard supervised, zero-shot cross-lingual, as well as few-shot cross-lingual settings for different natural language understanding tasks.
  arXiv  Detail & Related papers  (2020-03-05T16:07:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.