Hierarchical Neural Dynamic Policies

• URL: http://arxiv.org/abs/2107.05627v1
• Date: Mon, 12 Jul 2021 17:59:58 GMT
• Title: Hierarchical Neural Dynamic Policies
• Authors: Shikhar Bahl, Abhinav Gupta, Deepak Pathak
• Abstract summary: We tackle the problem of generalization to unseen configurations for dynamic tasks in the real world while learning from high-dimensional image input. We use hierarchical deep policy learning framework called Hierarchical Neural Dynamical Policies (H-NDPs) H-NDPs form a curriculum by learning local dynamical system-based policies on small regions in state-space. We show that H-NDPs are easily integrated with both imitation as well as reinforcement learning setups and achieve state-of-the-art results.
• Score: 50.969565411919376
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We tackle the problem of generalization to unseen configurations for dynamic tasks in the real world while learning from high-dimensional image input. The family of nonlinear dynamical system-based methods have successfully demonstrated dynamic robot behaviors but have difficulty in generalizing to unseen configurations as well as learning from image inputs. Recent works approach this issue by using deep network policies and reparameterize actions to embed the structure of dynamical systems but still struggle in domains with diverse configurations of image goals, and hence, find it difficult to generalize. In this paper, we address this dichotomy by leveraging embedding the structure of dynamical systems in a hierarchical deep policy learning framework, called Hierarchical Neural Dynamical Policies (H-NDPs). Instead of fitting deep dynamical systems to diverse data directly, H-NDPs form a curriculum by learning local dynamical system-based policies on small regions in state-space and then distill them into a global dynamical system-based policy that operates only from high-dimensional images. H-NDPs additionally provide smooth trajectories, a strong safety benefit in the real world. We perform extensive experiments on dynamic tasks both in the real world (digit writing, scooping, and pouring) and simulation (catching, throwing, picking). We show that H-NDPs are easily integrated with both imitation as well as reinforcement learning setups and achieve state-of-the-art results. Video results are at https://shikharbahl.github.io/hierarchical-ndps/

Related papers

• Learning System Dynamics without Forgetting [60.08612207170659]
  Predicting trajectories of systems with unknown dynamics is crucial in various research fields, including physics and biology. We present a novel framework of Mode-switching Graph ODE (MS-GODE), which can continually learn varying dynamics. We construct a novel benchmark of biological dynamic systems, featuring diverse systems with disparate dynamics.
  arXiv  Detail & Related papers  (2024-06-30T14:55:18Z)
• Efficient PAC Learnability of Dynamical Systems Over Multilayer Networks [30.424671907681688]
  We study the learnability of dynamical systems over multilayer networks, which are more realistic and challenging. We present an efficient PAC learning algorithm with provable guarantees to show that the learner only requires a small number of training examples to infer an unknown system.
  arXiv  Detail & Related papers  (2024-05-11T02:35:08Z)
• On the effectiveness of neural priors in modeling dynamical systems [28.69155113611877]
  We discuss the architectural regularization that neural networks offer when learning such systems. We show that simple coordinate networks with few layers can be used to solve multiple problems in modelling dynamical systems.
  arXiv  Detail & Related papers  (2023-03-10T06:21:24Z)
• Quantification before Selection: Active Dynamics Preference for Robust Reinforcement Learning [5.720802072821204]
  We introduce Active Dynamics Preference(ADP), which quantifies the informativeness and density of sampled system parameters. We validate our approach in four robotic locomotion tasks with various discrepancies between the training and testing environments.
  arXiv  Detail & Related papers  (2022-09-23T13:59:55Z)
• D$^3$FlowSLAM: Self-Supervised Dynamic SLAM with Flow Motion Decomposition and DINO Guidance [61.14088096348959]
  We introduce a self-supervised deep SLAM method that robustly operates in dynamic scenes while accurately identifying dynamic components. We propose a dynamic update module based on this representation and develop a dense SLAM system that excels in dynamic scenarios.
  arXiv  Detail & Related papers  (2022-07-18T17:47:39Z)
• An Efficient Image-to-Image Translation HourGlass-based Architecture for Object Pushing Policy Learning [20.77172985076276]
  Humans effortlessly solve pushing tasks in everyday life but unlocking these capabilities remains a challenge in robotics. We present an architecture combining a predictor of which pushes lead to changes in the environment with a state-action value predictor dedicated to the pushing task. We demonstrate in simulation experiments with a UR5 robot arm that our overall architecture helps the DQN learn faster and achieve higher performance.
  arXiv  Detail & Related papers  (2021-08-02T16:46:08Z)
• Composable Learning with Sparse Kernel Representations [110.19179439773578]
  We present a reinforcement learning algorithm for learning sparse non-parametric controllers in a Reproducing Kernel Hilbert Space. We improve the sample complexity of this approach by imposing a structure of the state-action function through a normalized advantage function. We demonstrate the performance of this algorithm on learning obstacle-avoidance policies in multiple simulations of a robot equipped with a laser scanner while navigating in a 2D environment.
  arXiv  Detail & Related papers  (2021-03-26T13:58:23Z)
• Neural Dynamic Policies for End-to-End Sensorimotor Learning [51.24542903398335]
  The current dominant paradigm in sensorimotor control, whether imitation or reinforcement learning, is to train policies directly in raw action spaces. We propose Neural Dynamic Policies (NDPs) that make predictions in trajectory distribution space. NDPs outperform the prior state-of-the-art in terms of either efficiency or performance across several robotic control tasks.
  arXiv  Detail & Related papers  (2020-12-04T18:59:32Z)
• Learning Stable Deep Dynamics Models [91.90131512825504]
  We propose an approach for learning dynamical systems that are guaranteed to be stable over the entire state space. We show that such learning systems are able to model simple dynamical systems and can be combined with additional deep generative models to learn complex dynamics.
  arXiv  Detail & Related papers  (2020-01-17T00:04:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.