Predicting the Future with Simple World Models
- URL: http://arxiv.org/abs/2401.17835v1
- Date: Wed, 31 Jan 2024 13:52:11 GMT
- Title: Predicting the Future with Simple World Models
- Authors: Tankred Saanum, Peter Dayan, Eric Schulz
- Abstract summary: We propose a regularization scheme that simplifies the world model's latent dynamics.
We find that our regularization improves accuracy, generalization, and performance in downstream tasks.
- Score: 12.051527678467775
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: World models can represent potentially high-dimensional pixel observations in
compact latent spaces, making it tractable to model the dynamics of the
environment. However, the latent dynamics inferred by these models may still be
highly complex. Abstracting the dynamics of the environment with simple models
can have several benefits. If the latent dynamics are simple, the model may
generalize better to novel transitions, and discover useful latent
representations of environment states. We propose a regularization scheme that
simplifies the world model's latent dynamics. Our model, the Parsimonious
Latent Space Model (PLSM), minimizes the mutual information between latent
states and the dynamics that arise between them. This makes the dynamics softly
state-invariant, and the effects of the agent's actions more predictable. We
combine the PLSM with three different model classes used for i) future latent
state prediction, ii) video prediction, and iii) planning. We find that our
regularization improves accuracy, generalization, and performance in downstream
tasks.
Related papers
- Synthetic location trajectory generation using categorical diffusion
models [50.809683239937584]
Diffusion models (DPMs) have rapidly evolved to be one of the predominant generative models for the simulation of synthetic data.
We propose using DPMs for the generation of synthetic individual location trajectories (ILTs) which are sequences of variables representing physical locations visited by individuals.
arXiv Detail & Related papers (2024-02-19T15:57:39Z) - Improving Deep Dynamics Models for Autonomous Vehicles with Multimodal
Latent Mapping of Surfaces [23.023397401781757]
We propose a new approach that learns a surface-aware dynamics model by conditioning it on a latent variable vector.
A latent mapper is trained to update these latent variables during inference from multiple modalities.
We show that by using this model, the driving performance can be improved on varying and challenging surfaces.
arXiv Detail & Related papers (2023-03-21T11:21:31Z) - Learning Robust Dynamics through Variational Sparse Gating [18.476155786474358]
In environments with many objects, often only a small number of them are moving or interacting at the same time.
In this paper, we investigate integrating this inductive bias of sparse interactions into the latent dynamics of world models trained from pixels.
arXiv Detail & Related papers (2022-10-21T02:56:51Z) - Conditioned Human Trajectory Prediction using Iterative Attention Blocks [70.36888514074022]
We present a simple yet effective pedestrian trajectory prediction model aimed at pedestrians positions prediction in urban-like environments.
Our model is a neural-based architecture that can run several layers of attention blocks and transformers in an iterative sequential fashion.
We show that without explicit introduction of social masks, dynamical models, social pooling layers, or complicated graph-like structures, it is possible to produce on par results with SoTA models.
arXiv Detail & Related papers (2022-06-29T07:49:48Z) - Dream to Explore: Adaptive Simulations for Autonomous Systems [3.0664963196464448]
We tackle the problem of learning to control dynamical systems by applying Bayesian nonparametric methods.
By employing Gaussian processes to discover latent world dynamics, we mitigate common data efficiency issues observed in reinforcement learning.
Our algorithm jointly learns a world model and policy by optimizing a variational lower bound of a log-likelihood.
arXiv Detail & Related papers (2021-10-27T04:27:28Z) - Analysis of ODE2VAE with Examples [0.0]
Ordinary Differential Equation Variational Auto-Encoder (ODE2VAE) is a deep latent variable model.
We show that the model is able to learn meaningful latent representations to an extent without any supervision.
arXiv Detail & Related papers (2021-08-10T20:12:26Z) - TCL: Transformer-based Dynamic Graph Modelling via Contrastive Learning [87.38675639186405]
We propose a novel graph neural network approach, called TCL, which deals with the dynamically-evolving graph in a continuous-time fashion.
To the best of our knowledge, this is the first attempt to apply contrastive learning to representation learning on dynamic graphs.
arXiv Detail & Related papers (2021-05-17T15:33:25Z) - Autoregressive Dynamics Models for Offline Policy Evaluation and
Optimization [60.73540999409032]
We show that expressive autoregressive dynamics models generate different dimensions of the next state and reward sequentially conditioned on previous dimensions.
We also show that autoregressive dynamics models are useful for offline policy optimization by serving as a way to enrich the replay buffer.
arXiv Detail & Related papers (2021-04-28T16:48:44Z) - S2RMs: Spatially Structured Recurrent Modules [105.0377129434636]
We take a step towards exploiting dynamic structure that are capable of simultaneously exploiting both modular andtemporal structures.
We find our models to be robust to the number of available views and better capable of generalization to novel tasks without additional training.
arXiv Detail & Related papers (2020-07-13T17:44:30Z) - Context-aware Dynamics Model for Generalization in Model-Based
Reinforcement Learning [124.9856253431878]
We decompose the task of learning a global dynamics model into two stages: (a) learning a context latent vector that captures the local dynamics, then (b) predicting the next state conditioned on it.
In order to encode dynamics-specific information into the context latent vector, we introduce a novel loss function that encourages the context latent vector to be useful for predicting both forward and backward dynamics.
The proposed method achieves superior generalization ability across various simulated robotics and control tasks, compared to existing RL schemes.
arXiv Detail & Related papers (2020-05-14T08:10:54Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.