QuadrupedGPT: Towards a Versatile Quadruped Agent in Open-ended Worlds

• URL: http://arxiv.org/abs/2406.16578v2
• Date: Tue, 03 Dec 2024 03:49:24 GMT
• Title: QuadrupedGPT: Towards a Versatile Quadruped Agent in Open-ended Worlds
• Authors: Yuting Mei, Ye Wang, Sipeng Zheng, Qin Jin,
• Abstract summary: We introduce QuadrupedGPT, designed to follow diverse commands with agility comparable to that of a pet. Our agent shows proficiency in handling diverse tasks and intricate instructions, representing a significant step toward the development of versatile quadruped agents.
• Score: 51.05639500325598
• License:
• Abstract: As robotic agents increasingly assist humans in reality, quadruped robots offer unique opportunities for interaction in complex scenarios due to their agile movement. However, building agents that can autonomously navigate, adapt, and respond to versatile goals remains a significant challenge. In this work, we introduce QuadrupedGPT designed to follow diverse commands with agility comparable to that of a pet. The primary challenges addressed include: i) effectively utilizing multimodal observations for informed decision-making; ii) achieving agile control by integrating locomotion and navigation; iii) developing advanced cognition to execute long-term objectives. Our QuadrupedGPT interprets human commands and environmental contexts using a large multimodal model. Leveraging its extensive knowledge base, the agent autonomously assigns parameters for adaptive locomotion policies and devises safe yet efficient paths toward its goals. Additionally, it employs high-level reasoning to decompose long-term goals into a sequence of executable subgoals. Through comprehensive experiments, our agent shows proficiency in handling diverse tasks and intricate instructions, representing a significant step toward the development of versatile quadruped agents for open-ended environments.

Related papers

• Grounding Language Models in Autonomous Loco-manipulation Tasks [3.8363685417355557]
  We propose a novel framework that learns, selects, and plans behaviors based on tasks in different scenarios. We leverage the planning and reasoning features of the large language model (LLM), constructing a hierarchical task graph. Experiments in simulation and real-world using the CENTAURO robot show that the language model based planner can efficiently adapt to new loco-manipulation tasks.
  arXiv  Detail & Related papers  (2024-09-02T15:27:48Z)
• Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control [106.32794844077534]
  This paper presents a study on using deep reinforcement learning to create dynamic locomotion controllers for bipedal robots. We develop a general control solution that can be used for a range of dynamic bipedal skills, from periodic walking and running to aperiodic jumping and standing. This work pushes the limits of agility for bipedal robots through extensive real-world experiments.
  arXiv  Detail & Related papers  (2024-01-30T10:48:43Z)
• Learning Diverse Skills for Local Navigation under Multi-constraint Optimality [27.310655303502305]
  In this work, we take a constrained optimization viewpoint on the quality-diversity trade-off. We show that we can obtain diverse policies while imposing constraints on their value functions which are defined through distinct rewards. Our trained policies transfer well to the real 12-DoF quadruped robot, Solo12.
  arXiv  Detail & Related papers  (2023-10-03T21:21:21Z)
• RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunking [54.776890150458385]
  We develop an efficient system for training universal agents capable of multi-task manipulation skills. We are able to train a single agent capable of 12 unique skills, and demonstrate its generalization over 38 tasks. On average, RoboAgent outperforms prior methods by over 40% in unseen situations.
  arXiv  Detail & Related papers  (2023-09-05T03:14:39Z)
• AlphaBlock: Embodied Finetuning for Vision-Language Reasoning in Robot Manipulation [50.737355245505334]
  We propose a novel framework for learning high-level cognitive capabilities in robot manipulation tasks. The resulting dataset AlphaBlock consists of 35 comprehensive high-level tasks of multi-step text plans and paired observation.
  arXiv  Detail & Related papers  (2023-05-30T09:54:20Z)
• Robust and Versatile Bipedal Jumping Control through Reinforcement Learning [141.56016556936865]
  This work aims to push the limits of agility for bipedal robots by enabling a torque-controlled bipedal robot to perform robust and versatile dynamic jumps in the real world. We present a reinforcement learning framework for training a robot to accomplish a large variety of jumping tasks, such as jumping to different locations and directions. We develop a new policy structure that encodes the robot's long-term input/output (I/O) history while also providing direct access to a short-term I/O history.
  arXiv  Detail & Related papers  (2023-02-19T01:06:09Z)
• Planning Immediate Landmarks of Targets for Model-Free Skill Transfer across Agents [34.56191646231944]
  We propose PILoT, i.e., Planning Immediate Landmarks of Targets. PILoT learns a goal-conditioned state planner and distills a goal-planner to plan immediate landmarks in a model-free style. We show the power of PILoT on various transferring challenges, including few-shot transferring across action spaces and dynamics.
  arXiv  Detail & Related papers  (2022-12-18T08:03:21Z)
• Evolving Hierarchical Memory-Prediction Machines in Multi-Task Reinforcement Learning [4.030910640265943]
  Behavioural agents must generalize across a variety of environments and objectives over time. We use genetic programming to evolve highly-generalized agents capable of operating in six unique environments from the control literature. We show that emergent hierarchical structure in the evolving programs leads to multi-task agents that succeed by performing a temporal decomposition and encoding of the problem environments in memory.
  arXiv  Detail & Related papers  (2021-06-23T21:34:32Z)
• Guided Navigation from Multiple Viewpoints using Qualitative Spatial Reasoning [0.0]
  This work aims to develop algorithms capable of guiding a sensory deprived robot to a goal location. The main task considered in this work is, given a group of autonomous agents, the development and evaluation of algorithms capable of producing a set of high-level commands.
  arXiv  Detail & Related papers  (2020-11-03T00:34:26Z)
• Automatic Curriculum Learning through Value Disagreement [95.19299356298876]
  Continually solving new, unsolved tasks is the key to learning diverse behaviors. In the multi-task domain, where an agent needs to reach multiple goals, the choice of training goals can largely affect sample efficiency. We propose setting up an automatic curriculum for goals that the agent needs to solve. We evaluate our method across 13 multi-goal robotic tasks and 5 navigation tasks, and demonstrate performance gains over current state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-06-17T03:58:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.