GEM: Group Enhanced Model for Learning Dynamical Control Systems

• URL: http://arxiv.org/abs/2104.02844v1
• Date: Wed, 7 Apr 2021 01:08:18 GMT
• Title: GEM: Group Enhanced Model for Learning Dynamical Control Systems
• Authors: Philippe Hansen-Estruch, Wenling Shang, Lerrel Pinto, Pieter Abbeel, Stas Tiomkin
• Abstract summary: We build effective dynamical models that are amenable to sample-based learning. We show that learning the dynamics on a Lie algebra vector space is more effective than learning a direct state transition model. This work sheds light on a connection between learning of dynamics and Lie group properties, which opens doors for new research directions.
• Score: 78.56159072162103
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Learning the dynamics of a physical system wherein an autonomous agent operates is an important task. Often these systems present apparent geometric structures. For instance, the trajectories of a robotic manipulator can be broken down into a collection of its transitional and rotational motions, fully characterized by the corresponding Lie groups and Lie algebras. In this work, we take advantage of these structures to build effective dynamical models that are amenable to sample-based learning. We hypothesize that learning the dynamics on a Lie algebra vector space is more effective than learning a direct state transition model. To verify this hypothesis, we introduce the Group Enhanced Model (GEM). GEMs significantly outperform conventional transition models on tasks of long-term prediction, planning, and model-based reinforcement learning across a diverse suite of standard continuous-control environments, including Walker, Hopper, Reacher, Half-Cheetah, Inverted Pendulums, Ant, and Humanoid. Furthermore, plugging GEM into existing state of the art systems enhances their performance, which we demonstrate on the PETS system. This work sheds light on a connection between learning of dynamics and Lie group properties, which opens doors for new research directions and practical applications along this direction. Our code is publicly available at: https://tinyurl.com/GEMMBRL.

Related papers

• Learning System Dynamics without Forgetting [60.08612207170659]
  Predicting trajectories of systems with unknown dynamics is crucial in various research fields, including physics and biology. We present a novel framework of Mode-switching Graph ODE (MS-GODE), which can continually learn varying dynamics. We construct a novel benchmark of biological dynamic systems, featuring diverse systems with disparate dynamics.
  arXiv  Detail & Related papers  (2024-06-30T14:55:18Z)
• Learning spatio-temporal patterns with Neural Cellular Automata [0.0]
  We train NCA to learn complex dynamics from time series of images and PDE trajectories. We extend NCA to capture both transient and stable structures within the same system. Being able to learn arbitrary dynamics gives NCA great potential as a data driven modelling framework.
  arXiv  Detail & Related papers  (2023-10-23T11:16:32Z)
• PASTA: Pretrained Action-State Transformer Agents [10.654719072766495]
  Self-supervised learning has brought about a revolutionary paradigm shift in various computing domains. Recent approaches involve pre-training transformer models on vast amounts of unlabeled data. In reinforcement learning, researchers have recently adapted these approaches, developing models pre-trained on expert trajectories.
  arXiv  Detail & Related papers  (2023-07-20T15:09:06Z)
• Physics-Inspired Temporal Learning of Quadrotor Dynamics for Accurate Model Predictive Trajectory Tracking [76.27433308688592]
  Accurately modeling quadrotor's system dynamics is critical for guaranteeing agile, safe, and stable navigation. We present a novel Physics-Inspired Temporal Convolutional Network (PI-TCN) approach to learning quadrotor's system dynamics purely from robot experience. Our approach combines the expressive power of sparse temporal convolutions and dense feed-forward connections to make accurate system predictions.
  arXiv  Detail & Related papers  (2022-06-07T13:51:35Z)
• Which priors matter? Benchmarking models for learning latent dynamics [70.88999063639146]
  Several methods have proposed to integrate priors from classical mechanics into machine learning models. We take a sober look at the current capabilities of these models. We find that the use of continuous and time-reversible dynamics benefits models of all classes.
  arXiv  Detail & Related papers  (2021-11-09T23:48:21Z)
• An Adaptable Approach to Learn Realistic Legged Locomotion without Examples [38.81854337592694]
  This work proposes a generic approach for ensuring realism in locomotion by guiding the learning process with the spring-loaded inverted pendulum model as a reference. We present experimental results showing that even in a model-free setup, the learned policies can generate realistic and energy-efficient locomotion gaits for a bipedal and a quadrupedal robot.
  arXiv  Detail & Related papers  (2021-10-28T10:14:47Z)
• Trajectory-wise Multiple Choice Learning for Dynamics Generalization in Reinforcement Learning [137.39196753245105]
  We present a new model-based reinforcement learning algorithm that learns a multi-headed dynamics model for dynamics generalization. We incorporate context learning, which encodes dynamics-specific information from past experiences into the context latent vector. Our method exhibits superior zero-shot generalization performance across a variety of control tasks, compared to state-of-the-art RL methods.
  arXiv  Detail & Related papers  (2020-10-26T03:20:42Z)
• CLOUD: Contrastive Learning of Unsupervised Dynamics [19.091886595825947]
  We propose to learn forward and inverse dynamics in a fully unsupervised manner via contrastive estimation. We demonstrate the efficacy of our approach across a variety of tasks including goal-directed planning and imitation from observations.
  arXiv  Detail & Related papers  (2020-10-23T15:42:57Z)
• Continual Model-Based Reinforcement Learning with Hypernetworks [24.86684067407964]
  We propose a method that continually learns encountered dynamics in a sequence of tasks using task-conditional hypernetworks. Our method has three main attributes: first, it includes dynamics learning sessions that do not revisit training data from previous tasks, so it only needs to store the most recent fixed-size portion of the state transition experience. We show that HyperCRL is effective in continual model-based reinforcement learning in robot locomotion and manipulation scenarios.
  arXiv  Detail & Related papers  (2020-09-25T01:46:26Z)
• S2RMs: Spatially Structured Recurrent Modules [105.0377129434636]
  We take a step towards exploiting dynamic structure that are capable of simultaneously exploiting both modular andtemporal structures. We find our models to be robust to the number of available views and better capable of generalization to novel tasks without additional training.
  arXiv  Detail & Related papers  (2020-07-13T17:44:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.