Building Open-Ended Embodied Agent via Language-Policy Bidirectional
Adaptation
- URL: http://arxiv.org/abs/2401.00006v3
- Date: Tue, 6 Feb 2024 16:30:55 GMT
- Title: Building Open-Ended Embodied Agent via Language-Policy Bidirectional
Adaptation
- Authors: Shaopeng Zhai, Jie Wang, Tianyi Zhang, Fuxian Huang, Qi Zhang, Ming
Zhou, Jing Hou, Yu Qiao and Yu Liu
- Abstract summary: Building embodied agents on integrating Large Language Models (LLMs) and Reinforcement Learning (RL) have revolutionized human-AI interaction.
Existing research faces challenges in meeting the requirement of open-endedness.
We present OpenPAL, a co-training framework comprising two stages: fine-tuning a pre-trained LLM to translate human instructions into goals for planning, and goal-conditioned training a policy for decision-making.
- Score: 40.82919989450566
- License: http://creativecommons.org/publicdomain/zero/1.0/
- Abstract: Building embodied agents on integrating Large Language Models (LLMs) and
Reinforcement Learning (RL) have revolutionized human-AI interaction:
researchers can now leverage language instructions to plan decision-making for
open-ended tasks. However, existing research faces challenges in meeting the
requirement of open-endedness. They typically either train LLM/RL models to
adapt to a fixed counterpart, limiting exploration of novel skills and
hindering the efficacy of human-AI interaction. To this end, we present
OpenPAL, a co-training framework comprising two stages: (1) fine-tuning a
pre-trained LLM to translate human instructions into goals for planning, and
goal-conditioned training a policy for decision-making; (2) co-training to
align the LLM and policy, achieving instruction open-endedness. We conducted
experiments using Contra, an open-ended FPS game, demonstrating that an agent
trained with OpenPAL not only comprehends arbitrary instructions but also
exhibits efficient execution. These results suggest that OpenPAL holds the
potential to construct open-ended embodied agents in practical scenarios.
Related papers
- Personalized Wireless Federated Learning for Large Language Models [75.22457544349668]
Large Language Models (LLMs) have revolutionized natural language processing tasks.
Their deployment in wireless networks still face challenges, i.e., a lack of privacy and security protection mechanisms.
We introduce two personalized wireless federated fine-tuning methods with low communication overhead.
arXiv Detail & Related papers (2024-04-20T02:30:21Z) - Empowering Large Language Models on Robotic Manipulation with Affordance Prompting [23.318449345424725]
Large language models fail to interact with the physical world by generating control sequences properly.
Existing LLM-based approaches circumvent this problem by relying on additional pre-defined skills or pre-trained sub-policies.
We propose a framework called LLM+A(ffordance) where the LLM serves as both the sub-task planner and the motion controller.
arXiv Detail & Related papers (2024-04-17T03:06:32Z) - From Summary to Action: Enhancing Large Language Models for Complex
Tasks with Open World APIs [62.496139001509114]
We introduce a novel tool invocation pipeline designed to control massive real-world APIs.
This pipeline mirrors the human task-solving process, addressing complicated real-life user queries.
Empirical evaluations of our Sum2Act pipeline on the ToolBench benchmark show significant performance improvements.
arXiv Detail & Related papers (2024-02-28T08:42:23Z) - Large Language Models as Agents in Two-Player Games [12.303405412105187]
This paper delineates the parallels between the training methods of large language models (LLMs) and the strategies employed for the development of agents in two-player games.
We propose a re-conceptualization of LLM learning processes in terms of agent learning in language-based games.
arXiv Detail & Related papers (2024-02-12T21:44:32Z) - GLIDE-RL: Grounded Language Instruction through DEmonstration in RL [7.658523833511356]
Training efficient Reinforcement Learning (RL) agents grounded in natural language has been a long-standing challenge.
We present a novel algorithm, Grounded Language Instruction through DEmonstration in RL (GLIDE-RL) that introduces a teacher-instructor-student curriculum learning framework.
In this multi-agent framework, the teacher and the student agents learn simultaneously based on the student's current skill level.
arXiv Detail & Related papers (2024-01-03T17:32:13Z) - LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language
Models [56.25156596019168]
This paper introduces the LMRL-Gym benchmark for evaluating multi-turn RL for large language models (LLMs)
Our benchmark consists of 8 different language tasks, which require multiple rounds of language interaction and cover a range of tasks in open-ended dialogue and text games.
arXiv Detail & Related papers (2023-11-30T03:59:31Z) - Zero-Shot Goal-Directed Dialogue via RL on Imagined Conversations [70.7884839812069]
Large language models (LLMs) have emerged as powerful and general solutions to many natural language tasks.
However, many of the most important applications of language generation are interactive, where an agent has to talk to a person to reach a desired outcome.
In this work, we explore a new method for adapting LLMs with RL for such goal-directed dialogue.
arXiv Detail & Related papers (2023-11-09T18:45:16Z) - Learning to Solve Voxel Building Embodied Tasks from Pixels and Natural
Language Instructions [53.21504989297547]
We propose a new method that combines a language model and reinforcement learning for the task of building objects in a Minecraft-like environment.
Our method first generates a set of consistently achievable sub-goals from the instructions and then completes associated sub-tasks with a pre-trained RL policy.
arXiv Detail & Related papers (2022-11-01T18:30:42Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.