Dropout's Dream Land: Generalization from Learned Simulators to Reality

• URL: http://arxiv.org/abs/2109.08342v1
• Date: Fri, 17 Sep 2021 03:58:56 GMT
• Title: Dropout's Dream Land: Generalization from Learned Simulators to Reality
• Authors: Zac Wellmer, James T. Kwok
• Abstract summary: A World Model is a generative model used to simulate an environment. In this work we explore improving the generalization capabilities from dream environments to real environments. We present a general approach to improve a controller's ability to transfer from a neural network dream environment to reality.
• Score: 33.9093915440877
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A World Model is a generative model used to simulate an environment. World Models have proven capable of learning spatial and temporal representations of Reinforcement Learning environments. In some cases, a World Model offers an agent the opportunity to learn entirely inside of its own dream environment. In this work we explore improving the generalization capabilities from dream environments to real environments (Dream2Real). We present a general approach to improve a controller's ability to transfer from a neural network dream environment to reality at little additional cost. These improvements are gained by drawing on inspiration from Domain Randomization, where the basic idea is to randomize as much of a simulator as possible without fundamentally changing the task at hand. Generally, Domain Randomization assumes access to a pre-built simulator with configurable parameters but oftentimes this is not available. By training the World Model using dropout, the dream environment is capable of creating a nearly infinite number of different dream environments. Previous use cases of dropout either do not use dropout at inference time or averages the predictions generated by multiple sampled masks (Monte-Carlo Dropout). Dropout's Dream Land leverages each unique mask to create a diverse set of dream environments. Our experimental results show that Dropout's Dream Land is an effective technique to bridge the reality gap between dream environments and reality. Furthermore, we additionally perform an extensive set of ablation studies.

Related papers

• Learning autonomous driving from aerial imagery [67.06858775696453]
  Photogrammetric simulators allow the synthesis of novel views through the transformation of pre-generated assets into novel views. We use a Neural Radiance Field (NeRF) as an intermediate representation to synthesize novel views from the point of view of a ground vehicle.
  arXiv  Detail & Related papers  (2024-10-18T05:09:07Z)
• One-shot World Models Using a Transformer Trained on a Synthetic Prior [37.027893127637036]
  One-Shot World Model (OSWM) is a transformer world model that is learned in an in-context learning fashion from purely synthetic data. OSWM is able to quickly adapt to the dynamics of a simple grid world, as well as the CartPole gym and a custom control environment.
  arXiv  Detail & Related papers  (2024-09-21T09:39:32Z)
• WorldDreamer: Towards General World Models for Video Generation via Predicting Masked Tokens [75.02160668328425]
  We introduce WorldDreamer, a pioneering world model to foster a comprehensive comprehension of general world physics and motions. WorldDreamer frames world modeling as an unsupervised visual sequence modeling challenge. Our experiments show that WorldDreamer excels in generating videos across different scenarios, including natural scenes and driving environments.
  arXiv  Detail & Related papers  (2024-01-18T14:01:20Z)
• Learning Interactive Real-World Simulators [96.5991333400566]
  We explore the possibility of learning a universal simulator of real-world interaction through generative modeling. We use the simulator to train both high-level vision-language policies and low-level reinforcement learning policies. Video captioning models can benefit from training with simulated experience, opening up even wider applications.
  arXiv  Detail & Related papers  (2023-10-09T19:42:22Z)
• Hieros: Hierarchical Imagination on Structured State Space Sequence World Models [4.922995343278039]
  Hieros is a hierarchical policy that learns time abstracted world representations and imagines trajectories at multiple time scales in latent space. We show that our approach outperforms the state of the art in terms of mean and median normalized human score on the Atari 100k benchmark.
  arXiv  Detail & Related papers  (2023-10-08T13:52:40Z)
• DriveDreamer: Towards Real-world-driven World Models for Autonomous Driving [76.24483706445298]
  We introduce DriveDreamer, a world model entirely derived from real-world driving scenarios. In the initial phase, DriveDreamer acquires a deep understanding of structured traffic constraints, while the subsequent stage equips it with the ability to anticipate future states. DriveDreamer enables the generation of realistic and reasonable driving policies, opening avenues for interaction and practical applications.
  arXiv  Detail & Related papers  (2023-09-18T13:58:42Z)
• Mastering Atari with Discrete World Models [61.7688353335468]
  We introduce DreamerV2, a reinforcement learning agent that learns behaviors purely from predictions in the compact latent space of a powerful world model. DreamerV2 constitutes the first agent that achieves human-level performance on the Atari benchmark of 55 tasks by learning behaviors inside a separately trained world model.
  arXiv  Detail & Related papers  (2020-10-05T17:52:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.