OPEn: An Open-ended Physics Environment for Learning Without a Task
- URL: http://arxiv.org/abs/2110.06912v1
- Date: Wed, 13 Oct 2021 17:48:23 GMT
- Title: OPEn: An Open-ended Physics Environment for Learning Without a Task
- Authors: Chuang Gan, Abhishek Bhandwaldar, Antonio Torralba, Joshua B.
Tenenbaum, Phillip Isola
- Abstract summary: We will study if models of the world learned in an open-ended physics environment, without any specific tasks, can be reused for downstream physics reasoning tasks.
We build a benchmark Open-ended Physics ENvironment (OPEn) and also design several tasks to test learning representations in this environment explicitly.
We find that an agent using unsupervised contrastive learning for representation learning, and impact-driven learning for exploration, achieved the best results.
- Score: 132.6062618135179
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Humans have mental models that allow them to plan, experiment, and reason in
the physical world. How should an intelligent agent go about learning such
models? In this paper, we will study if models of the world learned in an
open-ended physics environment, without any specific tasks, can be reused for
downstream physics reasoning tasks. To this end, we build a benchmark
Open-ended Physics ENvironment (OPEn) and also design several tasks to test
learning representations in this environment explicitly. This setting reflects
the conditions in which real agents (i.e. rolling robots) find themselves,
where they may be placed in a new kind of environment and must adapt without
any teacher to tell them how this environment works. This setting is
challenging because it requires solving an exploration problem in addition to a
model building and representation learning problem. We test several existing
RL-based exploration methods on this benchmark and find that an agent using
unsupervised contrastive learning for representation learning, and
impact-driven learning for exploration, achieved the best results. However, all
models still fall short in sample efficiency when transferring to the
downstream tasks. We expect that OPEn will encourage the development of novel
rolling robot agents that can build reusable mental models of the world that
facilitate many tasks.
Related papers
- Reward-Free Curricula for Training Robust World Models [37.13175950264479]
Learning world models from reward-free exploration is a promising approach, and enables policies to be trained using imagined experience for new tasks.
We address the novel problem of generating curricula in the reward-free setting to train robust world models.
We show that minimax regret can be connected to minimising the maximum error in the world model across environment instances.
This result informs our algorithm, WAKER: Weighted Acquisition of Knowledge across Environments for Robustness.
arXiv Detail & Related papers (2023-06-15T15:40:04Z) - Investigating the role of model-based learning in exploration and
transfer [11.652741003589027]
In this paper, we investigate transfer learning in the context of model-based agents.
We find that a model-based approach outperforms controlled model-free baselines for transfer learning.
Our results show that intrinsic exploration combined with environment models present a viable direction towards agents that are self-supervised and able to generalize to novel reward functions.
arXiv Detail & Related papers (2023-02-08T11:49:58Z) - Predictive World Models from Real-World Partial Observations [66.80340484148931]
We present a framework for learning a probabilistic predictive world model for real-world road environments.
While prior methods require complete states as ground truth for learning, we present a novel sequential training method to allow HVAEs to learn to predict complete states from partially observed states only.
arXiv Detail & Related papers (2023-01-12T02:07:26Z) - Towards Autonomous Grading In The Real World [4.651327752886103]
We aim to tackle the problem of autonomous grading, where a dozer is required to flatten an uneven area.
We design both a realistic physical simulation and a scaled real prototype environment mimicking the real dozer dynamics and sensory information.
arXiv Detail & Related papers (2022-06-13T12:21:20Z) - Should Models Be Accurate? [14.044354912031864]
We focus our investigations on Dyna-style planning in a prediction setting.
We introduce a meta-learning algorithm for training models with a focus on their usefulness to the learner instead of their accuracy in modelling the environment.
Our experiments show that our algorithm enables faster learning than even using an accurate model built with domain-specific knowledge of the non-stationarity.
arXiv Detail & Related papers (2022-05-22T04:23:54Z) - Continual Predictive Learning from Videos [100.27176974654559]
We study a new continual learning problem in the context of video prediction.
We propose the continual predictive learning (CPL) approach, which learns a mixture world model via predictive experience replay.
We construct two new benchmarks based on RoboNet and KTH, in which different tasks correspond to different physical robotic environments or human actions.
arXiv Detail & Related papers (2022-04-12T08:32:26Z) - Multitask Adaptation by Retrospective Exploration with Learned World
Models [77.34726150561087]
We propose a meta-learned addressing model called RAMa that provides training samples for the MBRL agent taken from task-agnostic storage.
The model is trained to maximize the expected agent's performance by selecting promising trajectories solving prior tasks from the storage.
arXiv Detail & Related papers (2021-10-25T20:02:57Z) - Learning intuitive physics and one-shot imitation using
state-action-prediction self-organizing maps [0.0]
Humans learn by exploration and imitation, build causal models of the world, and use both to flexibly solve new tasks.
We suggest a simple but effective unsupervised model which develops such characteristics.
We demonstrate its performance on a set of several related, but different one-shot imitation tasks, which the agent flexibly solves in an active inference style.
arXiv Detail & Related papers (2020-07-03T12:29:11Z) - Ecological Reinforcement Learning [76.9893572776141]
We study the kinds of environment properties that can make learning under such conditions easier.
understanding how properties of the environment impact the performance of reinforcement learning agents can help us to structure our tasks in ways that make learning tractable.
arXiv Detail & Related papers (2020-06-22T17:55:03Z) - An Exploration of Embodied Visual Exploration [97.21890864063872]
Embodied computer vision considers perception for robots in novel, unstructured environments.
We present a taxonomy for existing visual exploration algorithms and create a standard framework for benchmarking them.
We then perform a thorough empirical study of the four state-of-the-art paradigms using the proposed framework.
arXiv Detail & Related papers (2020-01-07T17:40:32Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.