Contextual Latent-Movements Off-Policy Optimization for Robotic
Manipulation Skills
- URL: http://arxiv.org/abs/2010.13766v3
- Date: Fri, 11 Feb 2022 01:49:11 GMT
- Title: Contextual Latent-Movements Off-Policy Optimization for Robotic
Manipulation Skills
- Authors: Samuele Tosatto, Georgia Chalvatzaki, Jan Peters
- Abstract summary: We propose a novel view on handling the demonstrated trajectories for acquiring low-dimensional, non-linear latent dynamics.
We introduce a new contextual off-policy RL algorithm, named LAtent-Movements Policy Optimization (LAMPO)
LAMPO provides sample-efficient policies against common approaches in literature.
- Score: 41.140532647789456
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Parameterized movement primitives have been extensively used for imitation
learning of robotic tasks. However, the high-dimensionality of the parameter
space hinders the improvement of such primitives in the reinforcement learning
(RL) setting, especially for learning with physical robots. In this paper we
propose a novel view on handling the demonstrated trajectories for acquiring
low-dimensional, non-linear latent dynamics, using mixtures of probabilistic
principal component analyzers (MPPCA) on the movements' parameter space.
Moreover, we introduce a new contextual off-policy RL algorithm, named
LAtent-Movements Policy Optimization (LAMPO). LAMPO can provide gradient
estimates from previous experience using self-normalized importance sampling,
hence, making full use of samples collected in previous learning iterations.
These advantages combined provide a complete framework for sample-efficient
off-policy optimization of movement primitives for robot learning of
high-dimensional manipulation skills. Our experimental results conducted both
in simulation and on a real robot show that LAMPO provides sample-efficient
policies against common approaches in literature.
Related papers
- Hyperparameter Optimization for Driving Strategies Based on Reinforcement Learning [1.3654846342364308]
We use Efficient Global Optimization algorithm to train RL agent in a simulation environment.
There is a substantial increase of 4% when compared to existing manually tuned parameters.
arXiv Detail & Related papers (2024-07-19T12:40:08Z) - Machine Learning Optimized Approach for Parameter Selection in MESHFREE Simulations [0.0]
Meshfree simulation methods are emerging as compelling alternatives to conventional mesh-based approaches.
We provide a comprehensive overview of our research combining Machine Learning (ML) and Fraunhofer's MESHFREE software.
We introduce a novel ML-optimized approach, using active learning, regression trees, and visualization on MESHFREE simulation data.
arXiv Detail & Related papers (2024-03-20T15:29:59Z) - Boosting Inference Efficiency: Unleashing the Power of Parameter-Shared
Pre-trained Language Models [109.06052781040916]
We introduce a technique to enhance the inference efficiency of parameter-shared language models.
We also propose a simple pre-training technique that leads to fully or partially shared models.
Results demonstrate the effectiveness of our methods on both autoregressive and autoencoding PLMs.
arXiv Detail & Related papers (2023-10-19T15:13:58Z) - Using Implicit Behavior Cloning and Dynamic Movement Primitive to
Facilitate Reinforcement Learning for Robot Motion Planning [1.7901837062462316]
Reinforcement learning (RL) for motion planning of robots suffers from low efficiency in terms of slow training speed and poor generalizability.
We propose a novel RL-based framework that uses implicit behavior cloning (IBC) and dynamic movement primitive (DMP) to improve the training speed and generalizability of an off-policy RL agent.
arXiv Detail & Related papers (2023-07-29T19:46:09Z) - A dynamic Bayesian optimized active recommender system for
curiosity-driven Human-in-the-loop automated experiments [8.780395483188242]
We present the development of a new type of human in the loop experimental workflow, via a Bayesian optimized active recommender system (BOARS)
This work shows the utility of human-augmented machine learning approaches for curiosity-driven exploration of systems across experimental domains.
arXiv Detail & Related papers (2023-04-05T14:54:34Z) - Towards Learning Universal Hyperparameter Optimizers with Transformers [57.35920571605559]
We introduce the OptFormer, the first text-based Transformer HPO framework that provides a universal end-to-end interface for jointly learning policy and function prediction.
Our experiments demonstrate that the OptFormer can imitate at least 7 different HPO algorithms, which can be further improved via its function uncertainty estimates.
arXiv Detail & Related papers (2022-05-26T12:51:32Z) - Gradient-Based Trajectory Optimization With Learned Dynamics [80.41791191022139]
We use machine learning techniques to learn a differentiable dynamics model of the system from data.
We show that a neural network can model highly nonlinear behaviors accurately for large time horizons.
In our hardware experiments, we demonstrate that our learned model can represent complex dynamics for both the Spot and Radio-controlled (RC) car.
arXiv Detail & Related papers (2022-04-09T22:07:34Z) - Transformer Inertial Poser: Attention-based Real-time Human Motion
Reconstruction from Sparse IMUs [79.72586714047199]
We propose an attention-based deep learning method to reconstruct full-body motion from six IMU sensors in real-time.
Our method achieves new state-of-the-art results both quantitatively and qualitatively, while being simple to implement and smaller in size.
arXiv Detail & Related papers (2022-03-29T16:24:52Z) - Nonprehensile Riemannian Motion Predictive Control [57.295751294224765]
We introduce a novel Real-to-Sim reward analysis technique to reliably imagine and predict the outcome of taking possible actions for a real robotic platform.
We produce a closed-loop controller to reactively push objects in a continuous action space.
We observe that RMPC is robust in cluttered as well as occluded environments and outperforms the baselines.
arXiv Detail & Related papers (2021-11-15T18:50:04Z) - Dimensionality Reduction of Movement Primitives in Parameter Space [34.16700176918835]
Movement primitives are an important policy class for real-world robotics.
The high dimensionality of their parametrization makes the policy optimization expensive both in terms of samples and computation.
We propose the application of dimensionality reduction in the parameter space, identifying principal movements.
arXiv Detail & Related papers (2020-02-26T16:38:39Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.