Robot Skill Adaptation via Soft Actor-Critic Gaussian Mixture Models

• URL: http://arxiv.org/abs/2111.13129v1
• Date: Thu, 25 Nov 2021 15:36:11 GMT
• Title: Robot Skill Adaptation via Soft Actor-Critic Gaussian Mixture Models
• Authors: Iman Nematollahi, Erick Rosete-Beas, Adrian R\"ofer, Tim Welschehold, Abhinav Valada, Wolfram Burgard
• Abstract summary: A core challenge for an autonomous agent acting in the real world is to adapt its repertoire of skills to cope with its noisy perception and dynamics. To scale learning of skills to long-horizon tasks, robots should be able to learn and later refine their skills in a structured manner. We proposeSAC-GMM, a novel hybrid approach that learns robot skills through a dynamical system and adapts the learned skills in their own trajectory distribution space.
• Score: 29.34375999491465
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A core challenge for an autonomous agent acting in the real world is to adapt its repertoire of skills to cope with its noisy perception and dynamics. To scale learning of skills to long-horizon tasks, robots should be able to learn and later refine their skills in a structured manner through trajectories rather than making instantaneous decisions individually at each time step. To this end, we propose the Soft Actor-Critic Gaussian Mixture Model (SAC-GMM), a novel hybrid approach that learns robot skills through a dynamical system and adapts the learned skills in their own trajectory distribution space through interactions with the environment. Our approach combines classical robotics techniques of learning from demonstration with the deep reinforcement learning framework and exploits their complementary nature. We show that our method utilizes sensors solely available during the execution of preliminarily learned skills to extract relevant features that lead to faster skill refinement. Extensive evaluations in both simulation and real-world environments demonstrate the effectiveness of our method in refining robot skills by leveraging physical interactions, high-dimensional sensory data, and sparse task completion rewards. Videos, code, and pre-trained models are available at \url{http://sac-gmm.cs.uni-freiburg.de}.

Related papers

• RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulation [68.70755196744533]
  RoboGen is a generative robotic agent that automatically learns diverse robotic skills at scale via generative simulation. Our work attempts to extract the extensive and versatile knowledge embedded in large-scale models and transfer them to the field of robotics.
  arXiv  Detail & Related papers  (2023-11-02T17:59:21Z)
• Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learning [58.3994826169858]
  We introduce RoboFuME, a reset-free fine-tuning system for robotic reinforcement learning. Our insights are to utilize offline reinforcement learning techniques to ensure efficient online fine-tuning of a pre-trained policy. Our method can incorporate data from an existing robot dataset and improve on a target task within as little as 3 hours of autonomous real-world experience.
  arXiv  Detail & Related papers  (2023-10-23T17:50:08Z)
• Robot Skill Generalization via Keypoint Integrated Soft Actor-Critic Gaussian Mixture Models [21.13906762261418]
  A long-standing challenge for a robotic manipulation system is adapting and generalizing its acquired motor skills to unseen environments. We tackle this challenge employing hybrid skill models that integrate imitation and reinforcement paradigms. We show that our method enables a robot to gain a significant zero-shot generalization to novel environments and to refine skills in the target environments faster than learning from scratch.
  arXiv  Detail & Related papers  (2023-10-23T16:03:23Z)
• Dexterous Manipulation from Images: Autonomous Real-World RL via Substep Guidance [71.36749876465618]
  We describe a system for vision-based dexterous manipulation that provides a "programming-free" approach for users to define new tasks. Our system includes a framework for users to define a final task and intermediate sub-tasks with image examples. experimental results with a four-finger robotic hand learning multi-stage object manipulation tasks directly in the real world.
  arXiv  Detail & Related papers  (2022-12-19T22:50:40Z)
• SAGCI-System: Towards Sample-Efficient, Generalizable, Compositional, and Incremental Robot Learning [41.19148076789516]
  We introduce a systematic learning framework called SAGCI-system towards achieving the above four requirements. Our system first takes the raw point clouds gathered by the camera mounted on the robot's wrist as the inputs and produces initial modeling of the surrounding environment represented as a URDF. The robot then utilizes the interactive perception to interact with the environments to online verify and modify the URDF.
  arXiv  Detail & Related papers  (2021-11-29T16:53:49Z)
• Bottom-Up Skill Discovery from Unsegmented Demonstrations for Long-Horizon Robot Manipulation [55.31301153979621]
  We tackle real-world long-horizon robot manipulation tasks through skill discovery. We present a bottom-up approach to learning a library of reusable skills from unsegmented demonstrations. Our method has shown superior performance over state-of-the-art imitation learning methods in multi-stage manipulation tasks.
  arXiv  Detail & Related papers  (2021-09-28T16:18:54Z)
• SKID RAW: Skill Discovery from Raw Trajectories [23.871402375721285]
  It is desirable to only demonstrate full task executions instead of all individual skills. We propose a novel approach that simultaneously learns to segment trajectories into reoccurring patterns. The approach learns a skill conditioning that can be used to understand possible sequences of skills.
  arXiv  Detail & Related papers  (2021-03-26T17:27:13Z)
• Cognitive architecture aided by working-memory for self-supervised multi-modal humans recognition [54.749127627191655]
  The ability to recognize human partners is an important social skill to build personalized and long-term human-robot interactions. Deep learning networks have achieved state-of-the-art results and demonstrated to be suitable tools to address such a task. One solution is to make robots learn from their first-hand sensory data with self-supervision.
  arXiv  Detail & Related papers  (2021-03-16T13:50:24Z)
• Learning and Sequencing of Object-Centric Manipulation Skills for Industrial Tasks [16.308562047398542]
  We propose a rapid robot skill-sequencing algorithm, where the skills are encoded by object-centric hidden semi-Markov models. The learned skill models can encode multimodal (temporal and spatial) trajectory distributions. We demonstrate this approach on a 7 DoF robot arm for industrial assembly tasks.
  arXiv  Detail & Related papers  (2020-08-24T14:20:05Z)
• Learning Agile Robotic Locomotion Skills by Imitating Animals [72.36395376558984]
  Reproducing the diverse and agile locomotion skills of animals has been a longstanding challenge in robotics. We present an imitation learning system that enables legged robots to learn agile locomotion skills by imitating real-world animals.
  arXiv  Detail & Related papers  (2020-04-02T02:56:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.