DreamingV2: Reinforcement Learning with Discrete World Models without Reconstruction

• URL: http://arxiv.org/abs/2203.00494v1
• Date: Tue, 1 Mar 2022 14:44:15 GMT
• Title: DreamingV2: Reinforcement Learning with Discrete World Models without Reconstruction
• Authors: Masashi Okada, Tadahiro Taniguchi
• Abstract summary: The present paper proposes a novel reinforcement learning method with world models, DreamingV2. DreamingV2 is a collaborative extension of DreamerV2 and Dreaming. We believe DreamingV2 will be a reliable solution for robot learning since its discrete representation is suitable to describe discontinuous environments.
• Score: 14.950054143767824
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The present paper proposes a novel reinforcement learning method with world models, DreamingV2, a collaborative extension of DreamerV2 and Dreaming. DreamerV2 is a cutting-edge model-based reinforcement learning from pixels that uses discrete world models to represent latent states with categorical variables. Dreaming is also a form of reinforcement learning from pixels that attempts to avoid the autoencoding process in general world model training by involving a reconstruction-free contrastive learning objective. The proposed DreamingV2 is a novel approach of adopting both the discrete representation of DreamingV2 and the reconstruction-free objective of Dreaming. Compared to DreamerV2 and other recent model-based methods without reconstruction, DreamingV2 achieves the best scores on five simulated challenging 3D robot arm tasks. We believe that DreamingV2 will be a reliable solution for robot learning since its discrete representation is suitable to describe discontinuous environments, and the reconstruction-free fashion well manages complex vision observations.

Related papers

• VistaDream: Sampling multiview consistent images for single-view scene reconstruction [63.991582576387856]
  VistaDream is a framework to reconstruct a 3D scene from a single-view image. Recent diffusion models enable generating high-quality novel-view images from a single-view input image.
  arXiv  Detail & Related papers  (2024-10-22T10:55:59Z)
• Dreaming is All You Need [6.706761900460658]
  This research introduces two novel deep learning models, SleepNet and DreamNet, to strike a balance between exploration and precision. SleepNet seamlessly integrates supervised learning with unsupervised sleep" stages using pre-trained encoder models. DreamNet employs full encoder-decoder frameworks to reconstruct the hidden states, mimicking the human "dreaming" process.
  arXiv  Detail & Related papers  (2024-09-03T06:04:39Z)
• MuDreamer: Learning Predictive World Models without Reconstruction [58.0159270859475]
  We present MuDreamer, a robust reinforcement learning agent that builds upon the DreamerV3 algorithm by learning a predictive world model without the need for reconstructing input signals. Our method achieves comparable performance on the Atari100k benchmark while benefiting from faster training.
  arXiv  Detail & Related papers  (2024-05-23T22:09:01Z)
• DreamCreature: Crafting Photorealistic Virtual Creatures from Imagination [140.1641573781066]
  We introduce a novel task, Virtual Creatures Generation: Given a set of unlabeled images of the target concepts, we aim to train a T2I model capable of creating new, hybrid concepts. We propose a new method called DreamCreature, which identifies and extracts the underlying sub-concepts. The T2I thus adapts to generate novel concepts with faithful structures and photorealistic appearance.
  arXiv  Detail & Related papers  (2023-11-27T01:24:31Z)
• DriveDreamer: Towards Real-world-driven World Models for Autonomous Driving [76.24483706445298]
  We introduce DriveDreamer, a world model entirely derived from real-world driving scenarios. In the initial phase, DriveDreamer acquires a deep understanding of structured traffic constraints, while the subsequent stage equips it with the ability to anticipate future states. DriveDreamer enables the generation of realistic and reasonable driving policies, opening avenues for interaction and practical applications.
  arXiv  Detail & Related papers  (2023-09-18T13:58:42Z)
• Dropout's Dream Land: Generalization from Learned Simulators to Reality [33.9093915440877]
  A World Model is a generative model used to simulate an environment. In this work we explore improving the generalization capabilities from dream environments to real environments. We present a general approach to improve a controller's ability to transfer from a neural network dream environment to reality.
  arXiv  Detail & Related papers  (2021-09-17T03:58:56Z)
• Mastering Atari with Discrete World Models [61.7688353335468]
  We introduce DreamerV2, a reinforcement learning agent that learns behaviors purely from predictions in the compact latent space of a powerful world model. DreamerV2 constitutes the first agent that achieves human-level performance on the Atari benchmark of 55 tasks by learning behaviors inside a separately trained world model.
  arXiv  Detail & Related papers  (2020-10-05T17:52:14Z)
• Dreaming: Model-based Reinforcement Learning by Latent Imagination without Reconstruction [14.950054143767824]
  We propose a decoder-free extension of Dreamer, a leading model-based reinforcement learning (MBRL) method from pixels. We derive a likelihood-free and InfoMax objective of contrastive learning from the evidence lower bound of Dreamer. In comparison to Dreamer and other recent model-free reinforcement learning methods, our newly devised Dreamer with InfoMax and without generative decoder (Dreaming) achieves the best scores on 5 difficult simulated robotics tasks.
  arXiv  Detail & Related papers  (2020-07-29T00:14:40Z)
• Large-Scale Adversarial Training for Vision-and-Language Representation Learning [81.76089876263175]
  We present VILLA, the first known effort on large-scale adversarial training for vision-and-language (V+L) representation learning. VILLA consists of two training stages: (i) task-agnostic adversarial pre-training; followed by (ii) task-specific adversarial finetuning.
  arXiv  Detail & Related papers  (2020-06-11T05:14:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.