Deep Sets for Generalization in RL

• URL: http://arxiv.org/abs/2003.09443v1
• Date: Fri, 20 Mar 2020 18:22:40 GMT
• Title: Deep Sets for Generalization in RL
• Authors: Tristan Karch, C\'edric Colas, Laetitia Teodorescu, Cl\'ement Moulin-Frier and Pierre-Yves Oudeyer
• Abstract summary: This paper investigates the idea of encoding object-centered representations in the design of the reward function and policy architectures of a language-guided reinforcement learning agent. In a 2D procedurally-generated world where agents targeting goals in natural language navigate and interact with objects, we show that these architectures demonstrate strong generalization capacities to out-of-distribution goals.
• Score: 15.092941080981706
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper investigates the idea of encoding object-centered representations in the design of the reward function and policy architectures of a language-guided reinforcement learning agent. This is done using a combination of object-wise permutation invariant networks inspired from Deep Sets and gated-attention mechanisms. In a 2D procedurally-generated world where agents targeting goals in natural language navigate and interact with objects, we show that these architectures demonstrate strong generalization capacities to out-of-distribution goals. We study the generalization to varying numbers of objects at test time and further extend the object-centered architectures to goals involving relational reasoning.

Related papers

• Learning to Compose: Improving Object Centric Learning by Injecting Compositionality [27.364435779446072]
  compositional representation is a key aspect of object-centric learning. Most of the existing approaches rely on auto-encoding objective. We propose a novel objective that explicitly encourages compositionality of the representations.
  arXiv  Detail & Related papers  (2024-05-01T17:21:36Z)
• Neural Slot Interpreters: Grounding Object Semantics in Emergent Slot Representations [4.807052027638089]
  We present the Neural Slot Interpreter (NSI) that learns to ground and generate object semantics via slot representations. NSI is an XML-like programming language that uses simple syntax rules to organize the object semantics of a scene into object-centric program primitives.
  arXiv  Detail & Related papers  (2024-02-02T12:37:23Z)
• Neural Constraint Satisfaction: Hierarchical Abstraction for Combinatorial Generalization in Object Rearrangement [75.9289887536165]
  We present a hierarchical abstraction approach to uncover underlying entities. We show how to learn a correspondence between intervening on states of entities in the agent's model and acting on objects in the environment. We use this correspondence to develop a method for control that generalizes to different numbers and configurations of objects.
  arXiv  Detail & Related papers  (2023-03-20T18:19:36Z)
• Discrete Factorial Representations as an Abstraction for Goal Conditioned Reinforcement Learning [99.38163119531745]
  We show that applying a discretizing bottleneck can improve performance in goal-conditioned RL setups. We experimentally prove the expected return on out-of-distribution goals, while still allowing for specifying goals with expressive structure.
  arXiv  Detail & Related papers  (2022-11-01T03:31:43Z)
• Compositional Generalization in Grounded Language Learning via Induced Model Sparsity [81.38804205212425]
  We consider simple language-conditioned navigation problems in a grid world environment with disentangled observations. We design an agent that encourages sparse correlations between words in the instruction and attributes of objects, composing them together to find the goal. Our agent maintains a high level of performance on goals containing novel combinations of properties even when learning from a handful of demonstrations.
  arXiv  Detail & Related papers  (2022-07-06T08:46:27Z)
• Complex-Valued Autoencoders for Object Discovery [62.26260974933819]
  We propose a distributed approach to object-centric representations: the Complex AutoEncoder. We show that this simple and efficient approach achieves better reconstruction performance than an equivalent real-valued autoencoder on simple multi-object datasets. We also show that it achieves competitive unsupervised object discovery performance to a SlotAttention model on two datasets, and manages to disentangle objects in a third dataset where SlotAttention fails - all while being 7-70 times faster to train.
  arXiv  Detail & Related papers  (2022-04-05T09:25:28Z)
• Feature-Attending Recurrent Modules for Generalization in Reinforcement Learning [27.736730414205137]
  "Feature- Recurrent Modules" (FARM) is an architecture for learning state representations that relies on simple, broadly applicable inductive biases for spatial and temporal regularities. FARM learns a state representation that is distributed across multiple modules that each attend to capturing features with an expressive feature attention mechanism. We show that this improves an RL agents ability to generalize across object-centric tasks.
  arXiv  Detail & Related papers  (2021-12-15T12:48:12Z)
• Neural Entity Linking: A Survey of Models Based on Deep Learning [82.43751915717225]
  This survey presents a comprehensive description of recent neural entity linking (EL) systems developed since 2015. Its goal is to systemize design features of neural entity linking systems and compare their performance to the remarkable classic methods on common benchmarks. The survey touches on applications of entity linking, focusing on the recently emerged use-case of enhancing deep pre-trained masked language models.
  arXiv  Detail & Related papers  (2020-05-31T18:02:26Z)
• Look-into-Object: Self-supervised Structure Modeling for Object Recognition [71.68524003173219]
  We propose to "look into object" (explicitly yet intrinsically model the object structure) through incorporating self-supervisions. We show the recognition backbone can be substantially enhanced for more robust representation learning. Our approach achieves large performance gain on a number of benchmarks, including generic object recognition (ImageNet) and fine-grained object recognition tasks (CUB, Cars, Aircraft)
  arXiv  Detail & Related papers  (2020-03-31T12:22:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.