LARM: Large Auto-Regressive Model for Long-Horizon Embodied Intelligence

• URL: http://arxiv.org/abs/2405.17424v1
• Date: Mon, 27 May 2024 17:59:32 GMT
• Title: LARM: Large Auto-Regressive Model for Long-Horizon Embodied Intelligence
• Authors: Zhuoling Li, Xiaogang Xu, Zhenhua Xu, SerNam Lim, Hengshuang Zhao,
• Abstract summary: We introduce the large auto-regressive model (LARM) for embodied agents. LARM uses both text and multi-view images as input and predicts subsequent actions in an auto-regressive manner. Adopting a two-phase training regimen, LARM successfully harvests enchanted equipment in Minecraft.
• Score: 68.27280750612204
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Due to the need to interact with the real world, embodied agents are required to possess comprehensive prior knowledge, long-horizon planning capability, and a swift response speed. Despite recent large language model (LLM) based agents achieving promising performance, they still exhibit several limitations. For instance, the output of LLMs is a descriptive sentence, which is ambiguous when determining specific actions. To address these limitations, we introduce the large auto-regressive model (LARM). LARM leverages both text and multi-view images as input and predicts subsequent actions in an auto-regressive manner. To train LARM, we develop a novel data format named auto-regressive node transmission structure and assemble a corresponding dataset. Adopting a two-phase training regimen, LARM successfully harvests enchanted equipment in Minecraft, which demands significantly more complex decision-making chains than the highest achievements of prior best methods. Besides, the speed of LARM is 6.8x faster.

Related papers

• On the Emergence of Thinking in LLMs I: Searching for the Right Intuition [34.32871896067864]
  We propose a post-training framework called Reinforcement Learning via Self-Play (RLSP) RLSP involves three steps: supervised fine-tuning with human or synthetic demonstrations of the reasoning process, using an exploration reward signal to encourage diverse and efficient reasoning behaviors, and RL training with an outcome verifier to ensure correctness while preventing reward hacking. Empirical studies in the math domain show that RLSP improves reasoning.
  arXiv  Detail & Related papers  (2025-02-10T18:52:04Z)
• Should You Use Your Large Language Model to Explore or Exploit? [55.562545113247666]
  We evaluate the ability of large language models to help a decision-making agent facing an exploration-exploitation tradeoff. We find that while the current LLMs often struggle to exploit, in-context mitigations may be used to substantially improve performance for small-scale tasks.
  arXiv  Detail & Related papers  (2025-01-31T23:42:53Z)
• Towards Efficient LLM Grounding for Embodied Multi-Agent Collaboration [70.09561665520043]
  We propose a novel framework for multi-agent collaboration that introduces Reinforced Advantage feedback (ReAd) for efficient self-refinement of plans. We provide theoretical analysis by extending advantage-weighted regression in reinforcement learning to multi-agent systems. Experiments on Over-AI and a difficult variant of RoCoBench show that ReAd surpasses baselines in success rate, and also significantly decreases the interaction steps of agents.
  arXiv  Detail & Related papers  (2024-05-23T08:33:19Z)
• TeaMs-RL: Teaching LLMs to Generate Better Instruction Datasets via Reinforcement Learning [7.9961739811640244]
  Development of Large Language Models often confronts challenges stemming from heavy reliance on human annotators. In this work, we pivot to Reinforcement Learning -- but with a twist. We use RL to directly generate the foundational instruction dataset that alone suffices for fine-tuning.
  arXiv  Detail & Related papers  (2024-03-13T16:57:57Z)
• Teaching Large Language Models to Reason with Reinforcement Learning [38.17625148525193]
  Reinforcement Learning from Human Feedback (textbfRLHF) has emerged as a dominant approach for aligning LLM outputs with human preferences. Inspired by the success of RLHF, we study the performance of multiple algorithms that learn from feedback.
  arXiv  Detail & Related papers  (2024-03-07T16:36:29Z)
• ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL [80.10358123795946]
  We develop a framework for building multi-turn RL algorithms for fine-tuning large language models. Our framework adopts a hierarchical RL approach and runs two RL algorithms in parallel. Empirically, we find that ArCHer significantly improves efficiency and performance on agent tasks.
  arXiv  Detail & Related papers  (2024-02-29T18:45:56Z)
• Reinforcement Learning from LLM Feedback to Counteract Goal Misgeneralization [0.0]
  We introduce a method to address goal misgeneralization in reinforcement learning (RL) Goal misgeneralization occurs when an agent retains its capabilities out-of-distribution yet pursues a proxy rather than the intended one. This study demonstrates how the Large Language Model can efficiently supervise RL agents.
  arXiv  Detail & Related papers  (2024-01-14T01:09:48Z)
• Language Reward Modulation for Pretraining Reinforcement Learning [61.76572261146311]
  We propose leveraging the capabilities of LRFs as a pretraining signal for reinforcement learning. Our VLM pretraining approach, which is a departure from previous attempts to use LRFs, can warmstart sample-efficient learning on robot manipulation tasks.
  arXiv  Detail & Related papers  (2023-08-23T17:37:51Z)
• Scalable Multi-Agent Inverse Reinforcement Learning via Actor-Attention-Critic [54.2180984002807]
  Multi-agent adversarial inverse reinforcement learning (MA-AIRL) is a recent approach that applies single-agent AIRL to multi-agent problems. We propose a multi-agent inverse RL algorithm that is more sample-efficient and scalable than previous works.
  arXiv  Detail & Related papers  (2020-02-24T20:30:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.