Probing Emergent Semantics in Predictive Agents via Question Answering
- URL: http://arxiv.org/abs/2006.01016v1
- Date: Mon, 1 Jun 2020 15:27:36 GMT
- Title: Probing Emergent Semantics in Predictive Agents via Question Answering
- Authors: Abhishek Das, Federico Carnevale, Hamza Merzic, Laura Rimell, Rosalia
Schneider, Josh Abramson, Alden Hung, Arun Ahuja, Stephen Clark, Gregory
Wayne, Felix Hill
- Abstract summary: Recent work has shown how predictive modeling can endow agents with rich knowledge of their surroundings, improving their ability to act in complex environments.
We propose question-answering as a general paradigm to decode and understand the representations that such agents develop the model.
We probe their internal state representations with synthetic (English) questions, without backpropagating gradients from the question-answering decoder into the agent.
- Score: 29.123837711842995
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Recent work has shown how predictive modeling can endow agents with rich
knowledge of their surroundings, improving their ability to act in complex
environments. We propose question-answering as a general paradigm to decode and
understand the representations that such agents develop, applying our method to
two recent approaches to predictive modeling -action-conditional CPC (Guo et
al., 2018) and SimCore (Gregor et al., 2019). After training agents with these
predictive objectives in a visually-rich, 3D environment with an assortment of
objects, colors, shapes, and spatial configurations, we probe their internal
state representations with synthetic (English) questions, without
backpropagating gradients from the question-answering decoder into the agent.
The performance of different agents when probed this way reveals that they
learn to encode factual, and seemingly compositional, information about
objects, properties and spatial relations from their physical environment. Our
approach is intuitive, i.e. humans can easily interpret responses of the model
as opposed to inspecting continuous vectors, and model-agnostic, i.e.
applicable to any modeling approach. By revealing the implicit knowledge of
objects, quantities, properties and relations acquired by agents as they learn,
question-conditional agent probing can stimulate the design and development of
stronger predictive learning objectives.
Related papers
- Evaluating the Utility of Model Explanations for Model Development [54.23538543168767]
We evaluate whether explanations can improve human decision-making in practical scenarios of machine learning model development.
To our surprise, we did not find evidence of significant improvement on tasks when users were provided with any of the saliency maps.
These findings suggest caution regarding the usefulness and potential for misunderstanding in saliency-based explanations.
arXiv Detail & Related papers (2023-12-10T23:13:23Z) - Understanding Your Agent: Leveraging Large Language Models for Behavior
Explanation [7.647395374489533]
We propose an approach to generate natural language explanations for an agent's behavior based only on observations of states and actions.
We show that our approach generates explanations as helpful as those produced by a human domain expert.
arXiv Detail & Related papers (2023-11-29T20:16:23Z) - Explaining Agent Behavior with Large Language Models [7.128139268426959]
We propose an approach to generate natural language explanations for an agent's behavior based only on observations of states and actions.
We show how a compact representation of the agent's behavior can be learned and used to produce plausible explanations.
arXiv Detail & Related papers (2023-09-19T06:13:24Z) - Knowledge-based Reasoning and Learning under Partial Observability in Ad
Hoc Teamwork [4.454557728745761]
This paper introduces an architecture that determines an ad hoc agent's behavior based on non-monotonic logical reasoning.
It supports online selection, adaptation, and learning of the models that predict the other agents' behavior.
We show that the performance of our architecture is comparable or better than state of the art data-driven baselines in both simple and complex scenarios.
arXiv Detail & Related papers (2023-06-01T15:21:27Z) - Planning for Learning Object Properties [117.27898922118946]
We formalize the problem of automatically training a neural network to recognize object properties as a symbolic planning problem.
We use planning techniques to produce a strategy for automating the training dataset creation and the learning process.
We provide an experimental evaluation in both a simulated and a real environment.
arXiv Detail & Related papers (2023-01-15T09:37:55Z) - Stochastic Coherence Over Attention Trajectory For Continuous Learning
In Video Streams [64.82800502603138]
This paper proposes a novel neural-network-based approach to progressively and autonomously develop pixel-wise representations in a video stream.
The proposed method is based on a human-like attention mechanism that allows the agent to learn by observing what is moving in the attended locations.
Our experiments leverage 3D virtual environments and they show that the proposed agents can learn to distinguish objects just by observing the video stream.
arXiv Detail & Related papers (2022-04-26T09:52:31Z) - Information is Power: Intrinsic Control via Information Capture [110.3143711650806]
We argue that a compact and general learning objective is to minimize the entropy of the agent's state visitation estimated using a latent state-space model.
This objective induces an agent to both gather information about its environment, corresponding to reducing uncertainty, and to gain control over its environment, corresponding to reducing the unpredictability of future world states.
arXiv Detail & Related papers (2021-12-07T18:50:42Z) - Modelling Behaviour Change using Cognitive Agent Simulations [0.0]
This paper presents work-in-progress research to apply selected behaviour change theories to simulated agents.
The research is focusing on complex agent architectures required for self-determined goal achievement in adverse circumstances.
arXiv Detail & Related papers (2021-10-16T19:19:08Z) - Backprop-Free Reinforcement Learning with Active Neural Generative
Coding [84.11376568625353]
We propose a computational framework for learning action-driven generative models without backpropagation of errors (backprop) in dynamic environments.
We develop an intelligent agent that operates even with sparse rewards, drawing inspiration from the cognitive theory of planning as inference.
The robust performance of our agent offers promising evidence that a backprop-free approach for neural inference and learning can drive goal-directed behavior.
arXiv Detail & Related papers (2021-07-10T19:02:27Z) - CausalCity: Complex Simulations with Agency for Causal Discovery and
Reasoning [68.74447489372037]
We present a high-fidelity simulation environment that is designed for developing algorithms for causal discovery and counterfactual reasoning.
A core component of our work is to introduce textitagency, such that it is simple to define and create complex scenarios.
We perform experiments with three state-of-the-art methods to create baselines and highlight the affordances of this environment.
arXiv Detail & Related papers (2021-06-25T00:21:41Z) - Learning intuitive physics and one-shot imitation using
state-action-prediction self-organizing maps [0.0]
Humans learn by exploration and imitation, build causal models of the world, and use both to flexibly solve new tasks.
We suggest a simple but effective unsupervised model which develops such characteristics.
We demonstrate its performance on a set of several related, but different one-shot imitation tasks, which the agent flexibly solves in an active inference style.
arXiv Detail & Related papers (2020-07-03T12:29:11Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.