Differentiable Rule Induction with Learned Relational Features

• URL: http://arxiv.org/abs/2201.06515v1
• Date: Mon, 17 Jan 2022 16:46:50 GMT
• Title: Differentiable Rule Induction with Learned Relational Features
• Authors: Remy Kusters, Yusik Kim, Marine Collery, Christian de Sainte Marie, Shubham Gupta
• Abstract summary: Rule Network (RRN) is a neural architecture that learns predicates that represent a linear relationship among attributes along with the rules that use them. On benchmark tasks we show that these predicates are simple enough to retain interpretability, yet improve prediction accuracy and provide sets of rules that are more concise compared to state of the art rule induction algorithms.
• Score: 9.193818627108572
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Rule-based decision models are attractive due to their interpretability. However, existing rule induction methods often results in long and consequently less interpretable set of rules. This problem can, in many cases, be attributed to the rule learner's lack of appropriately expressive vocabulary, i.e., relevant predicates. Most existing rule induction algorithms presume the availability of predicates used to represent the rules, naturally decoupling the predicate definition and the rule learning phases. In contrast, we propose the Relational Rule Network (RRN), a neural architecture that learns relational predicates that represent a linear relationship among attributes along with the rules that use them. This approach opens the door to increasing the expressiveness of induced decision models by coupling predicate learning directly with rule learning in an end to end differentiable fashion. On benchmark tasks, we show that these relational predicates are simple enough to retain interpretability, yet improve prediction accuracy and provide sets of rules that are more concise compared to state of the art rule induction algorithms.

Related papers

• Neuro-Symbolic Rule Lists [31.085257698392354]
  NeuRules is an end-to-end trainable model that unifies discretization, rule learning, and rule order into a single framework. We show that NeuRules consistently outperforms neuro-symbolic methods, effectively learning simple and complex rules, as well as their order, across a wide range of datasets.
  arXiv  Detail & Related papers  (2024-11-10T11:10:36Z)
• Rule Extrapolation in Language Models: A Study of Compositional Generalization on OOD Prompts [14.76420070558434]
  Rule extrapolation describes OOD scenarios, where the prompt violates at least one rule. We focus on formal languages, which are defined by the intersection of rules. We lay the first stones of a normative theory of rule extrapolation, inspired by the Solomonoff prior in algorithmic information theory.
  arXiv  Detail & Related papers  (2024-09-09T22:36:35Z)
• Can LLMs Reason with Rules? Logic Scaffolding for Stress-Testing and Improving LLMs [87.34281749422756]
  Large language models (LLMs) have achieved impressive human-like performance across various reasoning tasks. However, their mastery of underlying inferential rules still falls short of human capabilities. We propose a logic scaffolding inferential rule generation framework, to construct an inferential rule base, ULogic.
  arXiv  Detail & Related papers  (2024-02-18T03:38:51Z)
• Probabilistic Truly Unordered Rule Sets [4.169915659794567]
  We propose TURS, for Truly Unordered Rule Sets. We exploit the probabilistic properties of our rule sets, with the intuition of only allowing rules to overlap if they have similar probabilistic outputs. We benchmark against a wide range of rule-based methods and demonstrate that our method learns rule sets that have lower model complexity and highly competitive predictive performance.
  arXiv  Detail & Related papers  (2024-01-18T12:03:19Z)
• ChatRule: Mining Logical Rules with Large Language Models for Knowledge Graph Reasoning [107.61997887260056]
  We propose a novel framework, ChatRule, unleashing the power of large language models for mining logical rules over knowledge graphs. Specifically, the framework is initiated with an LLM-based rule generator, leveraging both the semantic and structural information of KGs. To refine the generated rules, a rule ranking module estimates the rule quality by incorporating facts from existing KGs.
  arXiv  Detail & Related papers  (2023-09-04T11:38:02Z)
• Learning Locally Interpretable Rule Ensemble [2.512827436728378]
  A rule ensemble is an interpretable model based on the linear combination of weighted rules. This paper proposes a new framework for learning a rule ensemble model that is both accurate and interpretable.
  arXiv  Detail & Related papers  (2023-06-20T12:06:56Z)
• RulE: Knowledge Graph Reasoning with Rule Embedding [69.31451649090661]
  We propose a principled framework called textbfRulE (stands for Rule Embedding) to leverage logical rules to enhance KG reasoning. RulE learns rule embeddings from existing triplets and first-order rules by jointly representing textbfentities, textbfrelations and textbflogical rules in a unified embedding space. Results on multiple benchmarks reveal that our model outperforms the majority of existing embedding-based and rule-based approaches.
  arXiv  Detail & Related papers  (2022-10-24T06:47:13Z)
• Truly Unordered Probabilistic Rule Sets for Multi-class Classification [0.0]
  We propose TURS, for Truly Unordered Rule Sets. We first formalise the problem of learning truly unordered rule sets. We then develop a two-phase algorithm that learns rule sets by carefully growing rules.
  arXiv  Detail & Related papers  (2022-06-17T14:34:35Z)
• Neuro-Symbolic Inductive Logic Programming with Logical Neural Networks [65.23508422635862]
  We propose learning rules with the recently proposed logical neural networks (LNN) Compared to others, LNNs offer strong connection to classical Boolean logic. Our experiments on standard benchmarking tasks confirm that LNN rules are highly interpretable.
  arXiv  Detail & Related papers  (2021-12-06T19:38:30Z)
• Learning Symbolic Rules for Reasoning in Quasi-Natural Language [74.96601852906328]
  We build a rule-based system that can reason with natural language input but without the manual construction of rules. We propose MetaQNL, a "Quasi-Natural" language that can express both formal logic and natural language sentences. Our approach achieves state-of-the-art accuracy on multiple reasoning benchmarks.
  arXiv  Detail & Related papers  (2021-11-23T17:49:00Z)
• RNNLogic: Learning Logic Rules for Reasoning on Knowledge Graphs [91.71504177786792]
  This paper studies learning logic rules for reasoning on knowledge graphs. Logic rules provide interpretable explanations when used for prediction as well as being able to generalize to other tasks. Existing methods either suffer from the problem of searching in a large search space or ineffective optimization due to sparse rewards.
  arXiv  Detail & Related papers  (2020-10-08T14:47:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.