Isolating and Leveraging Controllable and Noncontrollable Visual
Dynamics in World Models
- URL: http://arxiv.org/abs/2205.13817v1
- Date: Fri, 27 May 2022 08:07:39 GMT
- Title: Isolating and Leveraging Controllable and Noncontrollable Visual
Dynamics in World Models
- Authors: Minting Pan, Xiangming Zhu, Yunbo Wang, Xiaokang Yang
- Abstract summary: We present Iso-Dream, which improves the Dream-to-Control framework in two aspects.
First, by optimizing inverse dynamics, we encourage world model to learn controllable and noncontrollable sources.
Second, we optimize the behavior of the agent on the decoupled latent imaginations of the world model.
- Score: 65.97707691164558
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: World models learn the consequences of actions in vision-based interactive
systems. However, in practical scenarios such as autonomous driving, there
commonly exists noncontrollable dynamics independent of the action signals,
making it difficult to learn effective world models. To tackle this problem, we
present a novel reinforcement learning approach named Iso-Dream, which improves
the Dream-to-Control framework in two aspects. First, by optimizing the inverse
dynamics, we encourage the world model to learn controllable and
noncontrollable sources of spatiotemporal changes on isolated state transition
branches. Second, we optimize the behavior of the agent on the decoupled latent
imaginations of the world model. Specifically, to estimate state values, we
roll-out the noncontrollable states into the future and associate them with the
current controllable state. In this way, the isolation of dynamics sources can
greatly benefit long-horizon decision-making of the agent, such as a
self-driving car that can avoid potential risks by anticipating the movement of
other vehicles. Experiments show that Iso-Dream is effective in decoupling the
mixed dynamics and remarkably outperforms existing approaches in a wide range
of visual control and prediction domains.
Related papers
- Probing Multimodal LLMs as World Models for Driving [72.18727651074563]
This study focuses on the application of Multimodal Large Language Models (MLLMs) within the domain of autonomous driving.
We evaluate the capability of various MLLMs as world models for driving from the perspective of a fixed in-car camera.
Our results highlight a critical gap in the current capabilities of state-of-the-art MLLMs.
arXiv Detail & Related papers (2024-05-09T17:52:42Z) - SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Controllable Adversaries [94.84458417662407]
We introduce SAFE-SIM, a novel diffusion-based controllable closed-loop safety-critical simulation framework.
We develop a novel approach to simulate safety-critical scenarios through an adversarial term in the denoising process.
We validate our framework empirically using the NuScenes dataset, demonstrating improvements in both realism and controllability.
arXiv Detail & Related papers (2023-12-31T04:14:43Z) - Drive Anywhere: Generalizable End-to-end Autonomous Driving with
Multi-modal Foundation Models [114.69732301904419]
We present an approach to apply end-to-end open-set (any environment/scene) autonomous driving that is capable of providing driving decisions from representations queryable by image and text.
Our approach demonstrates unparalleled results in diverse tests while achieving significantly greater robustness in out-of-distribution situations.
arXiv Detail & Related papers (2023-10-26T17:56:35Z) - GAIA-1: A Generative World Model for Autonomous Driving [9.578453700755318]
We introduce GAIA-1 ('Generative AI for Autonomy'), a generative world model that generates realistic driving scenarios.
Emerging properties from our model include learning high-level structures and scene dynamics, contextual awareness, generalization, and understanding of geometry.
arXiv Detail & Related papers (2023-09-29T09:20:37Z) - Model-Based Reinforcement Learning with Isolated Imaginations [61.67183143982074]
We propose Iso-Dream++, a model-based reinforcement learning approach.
We perform policy optimization based on the decoupled latent imaginations.
This enables long-horizon visuomotor control tasks to benefit from isolating mixed dynamics sources in the wild.
arXiv Detail & Related papers (2023-03-27T02:55:56Z) - Active Learning of Discrete-Time Dynamics for Uncertainty-Aware Model
Predictive Control [49.60520501097199]
We present a self-supervised learning approach that actively models the dynamics of nonlinear robotic systems.
Our approach showcases high resilience and generalization capabilities by consistently adapting to unseen flight conditions.
arXiv Detail & Related papers (2022-10-23T00:45:05Z) - Dream to Explore: Adaptive Simulations for Autonomous Systems [3.0664963196464448]
We tackle the problem of learning to control dynamical systems by applying Bayesian nonparametric methods.
By employing Gaussian processes to discover latent world dynamics, we mitigate common data efficiency issues observed in reinforcement learning.
Our algorithm jointly learns a world model and policy by optimizing a variational lower bound of a log-likelihood.
arXiv Detail & Related papers (2021-10-27T04:27:28Z) - Cycle-Consistent World Models for Domain Independent Latent Imagination [0.0]
High costs and risks make it hard to train autonomous cars in the real world.
We propose a novel model-based reinforcement learning approach called Cycleconsistent World Models.
arXiv Detail & Related papers (2021-10-02T13:55:50Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.