Temporal Predictive Coding For Model-Based Planning In Latent Space
- URL: http://arxiv.org/abs/2106.07156v1
- Date: Mon, 14 Jun 2021 04:31:15 GMT
- Title: Temporal Predictive Coding For Model-Based Planning In Latent Space
- Authors: Tung Nguyen, Rui Shu, Tuan Pham, Hung Bui, Stefano Ermon
- Abstract summary: We present an information-theoretic approach that employs temporal predictive coding to encode elements in the environment that can be predicted across time.
We evaluate our model on a challenging modification of standard DMControl tasks where the background is replaced with natural videos that contain complex but irrelevant information to the planning task.
- Score: 80.99554006174093
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: High-dimensional observations are a major challenge in the application of
model-based reinforcement learning (MBRL) to real-world environments. To handle
high-dimensional sensory inputs, existing approaches use representation
learning to map high-dimensional observations into a lower-dimensional latent
space that is more amenable to dynamics estimation and planning. In this work,
we present an information-theoretic approach that employs temporal predictive
coding to encode elements in the environment that can be predicted across time.
Since this approach focuses on encoding temporally-predictable information, we
implicitly prioritize the encoding of task-relevant components over nuisance
information within the environment that are provably task-irrelevant. By
learning this representation in conjunction with a recurrent state space model,
we can then perform planning in latent space. We evaluate our model on a
challenging modification of standard DMControl tasks where the background is
replaced with natural videos that contain complex but irrelevant information to
the planning task. Our experiments show that our model is superior to existing
methods in the challenging complex-background setting while remaining
competitive with current state-of-the-art models in the standard setting.
Related papers
- EasyST: A Simple Framework for Spatio-Temporal Prediction [18.291117879544945]
We propose a simple framework for spatial-temporal prediction - EasyST paradigm.
It learns lightweight and robust Multi-Layer Perceptrons (MLPs) generalization by distilling knowledge from complex-temporal GNNs.
EasyST surpasses state-of-the-art approaches in terms of efficiency and accuracy.
arXiv Detail & Related papers (2024-09-10T11:40:01Z) - Learning Latent Dynamic Robust Representations for World Models [9.806852421730165]
Visual Model-Based Reinforcement Learning (MBL) promises to agent's knowledge about the underlying dynamics of the environment.
Top-temporal agents such as Dreamer often struggle with visual pixel-based inputs in the presence of irrelevant noise in the observation space.
We apply a-temporal masking strategy, combined with latent reconstruction, to capture endogenous task-specific aspects of the environment for world models.
arXiv Detail & Related papers (2024-05-10T06:28:42Z) - Predictive Experience Replay for Continual Visual Control and
Forecasting [62.06183102362871]
We present a new continual learning approach for visual dynamics modeling and explore its efficacy in visual control and forecasting.
We first propose the mixture world model that learns task-specific dynamics priors with a mixture of Gaussians, and then introduce a new training strategy to overcome catastrophic forgetting.
Our model remarkably outperforms the naive combinations of existing continual learning and visual RL algorithms on DeepMind Control and Meta-World benchmarks with continual visual control tasks.
arXiv Detail & Related papers (2023-03-12T05:08:03Z) - Spatio-temporal predictive tasks for abnormal event detection in videos [60.02503434201552]
We propose new constrained pretext tasks to learn object level normality patterns.
Our approach consists in learning a mapping between down-scaled visual queries and their corresponding normal appearance and motion characteristics.
Experiments on several benchmark datasets demonstrate the effectiveness of our approach to localize and track anomalies.
arXiv Detail & Related papers (2022-10-27T19:45:12Z) - NTFields: Neural Time Fields for Physics-Informed Robot Motion Planning [1.9798034349981157]
We propose Neural Time Fields (NTFields) for robot motion planning in cluttered scenarios.
Our framework represents a wave propagation model generating continuous arrival time to find path solutions informed by a nonlinear first-order PDE called Eikonal Equation.
We evaluate our method in various cluttered 3D environments, including the Gibson dataset, and demonstrate its ability to solve motion planning problems for 4-DOF and 6-DOF robot manipulators.
arXiv Detail & Related papers (2022-09-30T22:34:54Z) - Goal-Aware Prediction: Learning to Model What Matters [105.43098326577434]
One of the fundamental challenges in using a learned forward dynamics model is the mismatch between the objective of the learned model and that of the downstream planner or policy.
We propose to direct prediction towards task relevant information, enabling the model to be aware of the current task and encouraging it to only model relevant quantities of the state space.
We find that our method more effectively models the relevant parts of the scene conditioned on the goal, and as a result outperforms standard task-agnostic dynamics models and model-free reinforcement learning.
arXiv Detail & Related papers (2020-07-14T16:42:59Z) - Mutual Information Maximization for Robust Plannable Representations [82.83676853746742]
We present MIRO, an information theoretic representational learning algorithm for model-based reinforcement learning.
We show that our approach is more robust than reconstruction objectives in the presence of distractors and cluttered scenes.
arXiv Detail & Related papers (2020-05-16T21:58:47Z) - Predictive Coding for Locally-Linear Control [92.35650774524399]
High-dimensional observations and unknown dynamics are major challenges when applying optimal control to many real-world decision making tasks.
The Learning Controllable Embedding (LCE) framework addresses these challenges by embedding the observations into a lower dimensional latent space.
We show theoretically that explicit next-observation prediction can be replaced with predictive coding.
arXiv Detail & Related papers (2020-03-02T18:20:41Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.