Learning Relational Rules from Rewards

• URL: http://arxiv.org/abs/2203.13599v2
• Date: Mon, 28 Mar 2022 08:43:06 GMT
• Title: Learning Relational Rules from Rewards
• Abstract summary: We build a simple model of relational policy learning based on a function approximator developed in relational reinforcement learning (RRL) We trained and tested our model in three Atari games that required to consider an increasingly number of potential relations: Breakout, Pong and Demon Attack. In each game, our model was able to select adequate relational representations and build a relational policy incrementally.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Humans perceive the world in terms of objects and relations between them. In fact, for any given pair of objects, there is a myriad of relations that apply to them. How does the cognitive system learn which relations are useful to characterize the task at hand? And how can it use these representations to build a relational policy to interact effectively with the environment? In this paper we proposed that this problem can be understood through the lens of a sub-field of symbolic machine learning called relational reinforcement learning (RRL). To demonstrate the potential of our approach, we build a simple model of relational policy learning based on a function approximator developed in RRL. We trained and tested our model in three Atari games that required to consider an increasingly number of potential relations: Breakout, Pong and Demon Attack. In each game, our model was able to select adequate relational representations and build a relational policy incrementally. We discuss the relationship between our model with models of relational and analogical reasoning, as well as its limitations and future directions of research.

Related papers

• An Introduction to Causal Reinforcement Learning [58.680653905480284]
  Causal inference and reinforcement learning operate over different aspects of the same building block, counterfactual relations.<n>We put under a unifying treatment different modes of learning, including online, off-policy, and causal calculus learning.<n>Specifically, we introduce and discuss through causal lenses generalized policy learning, where to intervene, imitation learning, and counterfactual learning.
  arXiv  Detail & Related papers  (2026-06-23T05:28:33Z)
• Relational Structural Causal Models [52.818153111470394]
  We develop relational structural causal models, extending structural causal models to settings where objects and their relations vary.<n>We propose relational neural causal models, a provably correct approach that outperforms non-relational baselines on simulated traffic scenes.
  arXiv  Detail & Related papers  (2026-06-12T18:54:06Z)
• Fine-Tuning Dynamics of In-Context Factual Recall in Transformers [63.1821972855273]
  In-context learning -- performing tasks based on examples given in the prompt -- has emerged in large language models.<n>We study how incontext learning leverages factual knowledge recall.<n>We analyze the supervised fine-tuning dynamics of a one-layer transformer on IC-recall data.
  arXiv  Detail & Related papers  (2026-05-26T23:47:26Z)
• Learning Complete Topology-Aware Correlations Between Relations for Inductive Link Prediction [121.65152276851619]
  We show that semantic correlations between relations are inherently edge-level and entity-independent. We propose a novel subgraph-based method, namely TACO, to model Topology-Aware COrrelations between relations. To further exploit the potential of RCN, we propose Complete Common Neighbor induced subgraph.
  arXiv  Detail & Related papers  (2023-09-20T08:11:58Z)
• On Neural Architecture Inductive Biases for Relational Tasks [76.18938462270503]
  We introduce a simple architecture based on similarity-distribution scores which we name Compositional Network generalization (CoRelNet) We find that simple architectural choices can outperform existing models in out-of-distribution generalizations.
  arXiv  Detail & Related papers  (2022-06-09T16:24:01Z)
• Learning to Compose Visual Relations [100.45138490076866]
  We propose to represent each relation as an unnormalized density (an energy-based model) We show that such a factorized decomposition allows the model to both generate and edit scenes with multiple sets of relations more faithfully.
  arXiv  Detail & Related papers  (2021-11-17T18:51:29Z)
• Open Relation Modeling: Learning to Define Relations between Entities [24.04238065663009]
  We propose to teach machines to generate definition-like relation descriptions by letting them learn from definitions of entities. Specifically, we fine-tune Pre-trained Language Models (PLMs) to produce definitions conditioned on extracted entity pairs. We show that PLMs can select interpretable and informative reasoning paths by confidence estimation, and the selected path can guide PLMs to generate better relation descriptions.
  arXiv  Detail & Related papers  (2021-08-20T16:03:23Z)
• pix2rule: End-to-end Neuro-symbolic Rule Learning [84.76439511271711]
  This paper presents a complete neuro-symbolic method for processing images into objects, learning relations and logical rules. The main contribution is a differentiable layer in a deep learning architecture from which symbolic relations and rules can be extracted. We demonstrate that our model scales beyond state-of-the-art symbolic learners and outperforms deep relational neural network architectures.
  arXiv  Detail & Related papers  (2021-06-14T15:19:06Z)
• ZS-BERT: Towards Zero-Shot Relation Extraction with Attribute Representation Learning [10.609715843964263]
  We formulate the zero-shot relation extraction problem by incorporating the text description of seen and unseen relations. We propose a novel multi-task learning model, zero-shot BERT, to directly predict unseen relations without hand-crafted labeling and multiple pairwise attribute classifications. Experiments conducted on two well-known datasets exhibit that ZS-BERT can outperform existing methods by at least 13.54% improvement on F1 score.
  arXiv  Detail & Related papers  (2021-04-10T06:53:41Z)
• Behavior Priors for Efficient Reinforcement Learning [97.81587970962232]
  We consider how information and architectural constraints can be combined with ideas from the probabilistic modeling literature to learn behavior priors. We discuss how such latent variable formulations connect to related work on hierarchical reinforcement learning (HRL) and mutual information and curiosity based objectives. We demonstrate the effectiveness of our framework by applying it to a range of simulated continuous control domains.
  arXiv  Detail & Related papers  (2020-10-27T13:17:18Z)
• Relation-Guided Representation Learning [53.60351496449232]
  We propose a new representation learning method that explicitly models and leverages sample relations. Our framework well preserves the relations between samples. By seeking to embed samples into subspace, we show that our method can address the large-scale and out-of-sample problem.
  arXiv  Detail & Related papers  (2020-07-11T10:57:45Z)
• Learning Relation Ties with a Force-Directed Graph in Distant Supervised Relation Extraction [39.73191604776768]
  Relation ties, defined as the correlation and mutual exclusion between different relations, are critical for distant supervised relation extraction. Existing approaches model this property by greedily learning local dependencies. We propose a novel force-directed graph based relation extraction model to comprehensively learn relation ties.
  arXiv  Detail & Related papers  (2020-04-21T14:41:38Z)
• Learning Object Placements For Relational Instructions by Hallucinating Scene Representations [26.897316325189205]
  We present a convolutional neural network for estimating pixelwise object placement probabilities for a set of spatial relations from a single input image. Our method does not require ground truth data for the pixelwise relational probabilities or 3D models of the objects. Results obtained using real-world data and human-robot experiments demonstrate the effectiveness of our method.
  arXiv  Detail & Related papers  (2020-01-23T12:58:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.