Information is Power: Intrinsic Control via Information Capture
- URL: http://arxiv.org/abs/2112.03899v1
- Date: Tue, 7 Dec 2021 18:50:42 GMT
- Title: Information is Power: Intrinsic Control via Information Capture
- Authors: Nicholas Rhinehart, Jenny Wang, Glen Berseth, John D. Co-Reyes,
Danijar Hafner, Chelsea Finn, Sergey Levine
- Abstract summary: We argue that a compact and general learning objective is to minimize the entropy of the agent's state visitation estimated using a latent state-space model.
This objective induces an agent to both gather information about its environment, corresponding to reducing uncertainty, and to gain control over its environment, corresponding to reducing the unpredictability of future world states.
- Score: 110.3143711650806
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Humans and animals explore their environment and acquire useful skills even
in the absence of clear goals, exhibiting intrinsic motivation. The study of
intrinsic motivation in artificial agents is concerned with the following
question: what is a good general-purpose objective for an agent? We study this
question in dynamic partially-observed environments, and argue that a compact
and general learning objective is to minimize the entropy of the agent's state
visitation estimated using a latent state-space model. This objective induces
an agent to both gather information about its environment, corresponding to
reducing uncertainty, and to gain control over its environment, corresponding
to reducing the unpredictability of future world states. We instantiate this
approach as a deep reinforcement learning agent equipped with a deep
variational Bayes filter. We find that our agent learns to discover, represent,
and exercise control of dynamic objects in a variety of partially-observed
environments sensed with visual observations without extrinsic reward.
Related papers
Err
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.