Learn Proportional Derivative Controllable Latent Space from Pixels
- URL: http://arxiv.org/abs/2110.08239v1
- Date: Fri, 15 Oct 2021 17:47:07 GMT
- Title: Learn Proportional Derivative Controllable Latent Space from Pixels
- Authors: Weiyao Wang, Marin Kobilarov and Gregory D. Hager
- Abstract summary: We propose to introduce additional learning objectives to enforce that the learned latent space is proportional derivative controllable.
In execution time, the simple PD-controller can be applied directly to the latent space encoded from pixels, to produce simple and effective control.
- Score: 29.951834120368094
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Recent advances in latent space dynamics model from pixels show promising
progress in vision-based model predictive control (MPC). However, executing MPC
in real time can be challenging due to its intensive computational cost in each
timestep. We propose to introduce additional learning objectives to enforce
that the learned latent space is proportional derivative controllable. In
execution time, the simple PD-controller can be applied directly to the latent
space encoded from pixels, to produce simple and effective control to systems
with visual observations. We show that our method outperforms baseline methods
to produce robust goal reaching and trajectory tracking in various
environments.
Related papers
- Predictive Experience Replay for Continual Visual Control and
Forecasting [62.06183102362871]
We present a new continual learning approach for visual dynamics modeling and explore its efficacy in visual control and forecasting.
We first propose the mixture world model that learns task-specific dynamics priors with a mixture of Gaussians, and then introduce a new training strategy to overcome catastrophic forgetting.
Our model remarkably outperforms the naive combinations of existing continual learning and visual RL algorithms on DeepMind Control and Meta-World benchmarks with continual visual control tasks.
arXiv Detail & Related papers (2023-03-12T05:08:03Z) - In-Distribution Barrier Functions: Self-Supervised Policy Filters that
Avoid Out-of-Distribution States [84.24300005271185]
We propose a control filter that wraps any reference policy and effectively encourages the system to stay in-distribution with respect to offline-collected safe demonstrations.
Our method is effective for two different visuomotor control tasks in simulation environments, including both top-down and egocentric view settings.
arXiv Detail & Related papers (2023-01-27T22:28:19Z) - Model-Based Reinforcement Learning via Latent-Space Collocation [110.04005442935828]
We argue that it is easier to solve long-horizon tasks by planning sequences of states rather than just actions.
We adapt the idea of collocation, which has shown good results on long-horizon tasks in optimal control literature, to the image-based setting by utilizing learned latent state space models.
arXiv Detail & Related papers (2021-06-24T17:59:18Z) - Vision-Based Autonomous Drone Control using Supervised Learning in
Simulation [0.0]
We propose a vision-based control approach using Supervised Learning for autonomous navigation and landing of MAVs in indoor environments.
We trained a Convolutional Neural Network (CNN) that maps low resolution image and sensor input to high-level control commands.
Our approach requires shorter training times than similar Reinforcement Learning approaches and can potentially overcome the limitations of manual data collection faced by comparable Supervised Learning approaches.
arXiv Detail & Related papers (2020-09-09T13:45:41Z) - Online Reinforcement Learning Control by Direct Heuristic Dynamic
Programming: from Time-Driven to Event-Driven [80.94390916562179]
Time-driven learning refers to the machine learning method that updates parameters in a prediction model continuously as new data arrives.
It is desirable to prevent the time-driven dHDP from updating due to insignificant system event such as noise.
We show how the event-driven dHDP algorithm works in comparison to the original time-driven dHDP.
arXiv Detail & Related papers (2020-06-16T05:51:25Z) - NewtonianVAE: Proportional Control and Goal Identification from Pixels
via Physical Latent Spaces [9.711378389037812]
We introduce a latent dynamics learning framework that is uniquely designed to induce proportional controlability in the latent space.
We show that our learned dynamics model enables proportional control from pixels, dramatically simplifies and accelerates behavioural cloning of vision-based controllers, and provides interpretable goal discovery when applied to imitation learning of switching controllers from demonstration.
arXiv Detail & Related papers (2020-06-02T21:41:38Z) - Approximate Inverse Reinforcement Learning from Vision-based Imitation
Learning [34.5366377122507]
We present a method for obtaining an implicit objective function for vision-based navigation.
The proposed methodology relies on Imitation Learning, Model Predictive Control (MPC), and an interpretation technique used in Deep Neural Networks.
arXiv Detail & Related papers (2020-04-17T03:36:50Z) - Goal-Conditioned End-to-End Visuomotor Control for Versatile Skill
Primitives [89.34229413345541]
We propose a conditioning scheme which avoids pitfalls by learning the controller and its conditioning in an end-to-end manner.
Our model predicts complex action sequences based directly on a dynamic image representation of the robot motion.
We report significant improvements in task success over representative MPC and IL baselines.
arXiv Detail & Related papers (2020-03-19T15:04:37Z) - Predictive Coding for Locally-Linear Control [92.35650774524399]
High-dimensional observations and unknown dynamics are major challenges when applying optimal control to many real-world decision making tasks.
The Learning Controllable Embedding (LCE) framework addresses these challenges by embedding the observations into a lower dimensional latent space.
We show theoretically that explicit next-observation prediction can be replaced with predictive coding.
arXiv Detail & Related papers (2020-03-02T18:20:41Z) - Aggressive Perception-Aware Navigation using Deep Optical Flow Dynamics
and PixelMPC [21.81438321320149]
We introduce deep optical flow (DOF) dynamics, which is a combination of optical flow and robot dynamics.
Using the DOF dynamics, MPC explicitly incorporates the predicted movement of relevant pixels into the planned trajectory of a robot.
Our implementation of DOF is memory-efficient, data-efficient, and computationally cheap so that it can be computed in real-time for use in an MPC framework.
arXiv Detail & Related papers (2020-01-07T22:33:12Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.