Hard Attention Control By Mutual Information Maximization
- URL: http://arxiv.org/abs/2103.06371v1
- Date: Wed, 10 Mar 2021 22:38:28 GMT
- Title: Hard Attention Control By Mutual Information Maximization
- Authors: Himanshu Sahni and Charles Isbell
- Abstract summary: Biological agents have adopted the principle of attention to limit the rate of incoming information from the environment.
We propose an approach for learning how to control a hard attention window by maximizing the mutual information between the environment state and the attention location at each step.
- Score: 4.56877715768796
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Biological agents have adopted the principle of attention to limit the rate
of incoming information from the environment. One question that arises is if an
artificial agent has access to only a limited view of its surroundings, how can
it control its attention to effectively solve tasks? We propose an approach for
learning how to control a hard attention window by maximizing the mutual
information between the environment state and the attention location at each
step. The agent employs an internal world model to make predictions about its
state and focuses attention towards where the predictions may be wrong.
Attention is trained jointly with a dynamic memory architecture that stores
partial observations and keeps track of the unobserved state. We demonstrate
that our approach is effective in predicting the full state from a sequence of
partial observations. We also show that the agent's internal representation of
the surroundings, a live mental map, can be used for control in two partially
observable reinforcement learning tasks. Videos of the trained agent can be
found at https://sites.google.com/view/hard-attention-control.
Related papers
- Attention Schema in Neural Agents [66.43628974353683]
In cognitive neuroscience, Attention Theory (AST) supports the idea of distinguishing attention from AS.
AST predicts that an agent can use its own AS to also infer the states of other agents' attention.
We explore different ways in which attention and AS interact with each other.
arXiv Detail & Related papers (2023-05-27T05:40:34Z) - Guaranteed Discovery of Controllable Latent States with Multi-Step
Inverse Models [51.754160866582005]
Agent-Controllable State Discovery algorithm (AC-State)
Algorithm consists of a multi-step inverse model (predicting actions from distant observations) with an information bottleneck.
We demonstrate the discovery of controllable latent state in three domains: localizing a robot arm with distractions, exploring in a maze alongside other agents, and navigating in the Matterport house simulator.
arXiv Detail & Related papers (2022-07-17T17:06:52Z) - Stochastic Coherence Over Attention Trajectory For Continuous Learning
In Video Streams [64.82800502603138]
This paper proposes a novel neural-network-based approach to progressively and autonomously develop pixel-wise representations in a video stream.
The proposed method is based on a human-like attention mechanism that allows the agent to learn by observing what is moving in the attended locations.
Our experiments leverage 3D virtual environments and they show that the proposed agents can learn to distinguish objects just by observing the video stream.
arXiv Detail & Related papers (2022-04-26T09:52:31Z) - Information is Power: Intrinsic Control via Information Capture [110.3143711650806]
We argue that a compact and general learning objective is to minimize the entropy of the agent's state visitation estimated using a latent state-space model.
This objective induces an agent to both gather information about its environment, corresponding to reducing uncertainty, and to gain control over its environment, corresponding to reducing the unpredictability of future world states.
arXiv Detail & Related papers (2021-12-07T18:50:42Z) - Alignment Attention by Matching Key and Query Distributions [48.93793773929006]
This paper introduces alignment attention that explicitly encourages self-attention to match the distributions of the key and query within each head.
It is simple to convert any models with self-attention, including pre-trained ones, to the proposed alignment attention.
On a variety of language understanding tasks, we show the effectiveness of our method in accuracy, uncertainty estimation, generalization across domains, and robustness to adversarial attacks.
arXiv Detail & Related papers (2021-10-25T00:54:57Z) - Attention or memory? Neurointerpretable agents in space and time [0.0]
We design a model incorporating a self-attention mechanism that implements task-state representations in semantic feature-space.
To evaluate the agent's selective properties, we add a large volume of task-irrelevant features to observations.
In line with neuroscience predictions, self-attention leads to increased robustness to noise compared to benchmark models.
arXiv Detail & Related papers (2020-07-09T15:04:26Z) - Neuroevolution of Self-Interpretable Agents [11.171154483167514]
Inattentional blindness is the psychological phenomenon that causes one to miss things in plain sight.
Motivated by selective attention, we study the properties of artificial agents that perceive the world through the lens of a self-attention bottleneck.
arXiv Detail & Related papers (2020-03-18T11:40:35Z) - Mutual Information-based State-Control for Intrinsically Motivated
Reinforcement Learning [102.05692309417047]
In reinforcement learning, an agent learns to reach a set of goals by means of an external reward signal.
In the natural world, intelligent organisms learn from internal drives, bypassing the need for external signals.
We propose to formulate an intrinsic objective as the mutual information between the goal states and the controllable states.
arXiv Detail & Related papers (2020-02-05T19:21:20Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.