Neuro-Symbolic Rule Lists

• URL: http://arxiv.org/abs/2411.06428v1
• Date: Sun, 10 Nov 2024 11:10:36 GMT
• Title: Neuro-Symbolic Rule Lists
• Authors: Sascha Xu, Nils Philipp Walter, Jilles Vreeken,
• Abstract summary: NeuRules is an end-to-end trainable model that unifies discretization, rule learning, and rule order into a single framework. We show that NeuRules consistently outperforms neuro-symbolic methods, effectively learning simple and complex rules, as well as their order, across a wide range of datasets.
• Score: 31.085257698392354
• License:
• Abstract: Machine learning models deployed in sensitive areas such as healthcare must be interpretable to ensure accountability and fairness. Rule lists (if Age < 35 $\wedge$ Priors > 0 then Recidivism = True, else if Next Condition . . . ) offer full transparency, making them well-suited for high-stakes decisions. However, learning such rule lists presents significant challenges. Existing methods based on combinatorial optimization require feature pre-discretization and impose restrictions on rule size. Neuro-symbolic methods use more scalable continuous optimization yet place similar pre-discretization constraints and suffer from unstable optimization. To address the existing limitations, we introduce NeuRules, an end-to-end trainable model that unifies discretization, rule learning, and rule order into a single differentiable framework. We formulate a continuous relaxation of the rule list learning problem that converges to a strict rule list through temperature annealing. NeuRules learns both the discretizations of individual features, as well as their combination into conjunctive rules without any pre-processing or restrictions. Extensive experiments demonstrate that NeuRules consistently outperforms both combinatorial and neuro-symbolic methods, effectively learning simple and complex rules, as well as their order, across a wide range of datasets.

Related papers

• SoFA: Shielded On-the-fly Alignment via Priority Rule Following [90.32819418613407]
  This paper introduces a novel alignment paradigm, priority rule following, which defines rules as the primary control mechanism in each dialog. We present PriorityDistill, a semi-automated approach for distilling priority following signals from simulations to ensure robust rule integration and adherence.
  arXiv  Detail & Related papers  (2024-02-27T09:52:27Z)
• Toward Unified Controllable Text Generation via Regular Expression Instruction [56.68753672187368]
  Our paper introduces Regular Expression Instruction (REI), which utilizes an instruction-based mechanism to fully exploit regular expressions' advantages to uniformly model diverse constraints. Our method only requires fine-tuning on medium-scale language models or few-shot, in-context learning on large language models, and requires no further adjustment when applied to various constraint combinations.
  arXiv  Detail & Related papers  (2023-09-19T09:05:14Z)
• Reinforcement Logic Rule Learning for Temporal Point Processes [17.535382791003176]
  We propose a framework that can incrementally expand the explanatory temporal logic rule set to explain the occurrence of temporal events. The proposed algorithm alternates between a master problem, where the current rule set weights are updated, and a subproblem, where a new rule is searched and included to best increase the likelihood. We evaluate our methods on both synthetic and real healthcare datasets, obtaining promising results.
  arXiv  Detail & Related papers  (2023-08-11T12:05:32Z)
• On Regularization and Inference with Label Constraints [62.60903248392479]
  We compare two strategies for encoding label constraints in a machine learning pipeline, regularization with constraints and constrained inference. For regularization, we show that it narrows the generalization gap by precluding models that are inconsistent with the constraints. For constrained inference, we show that it reduces the population risk by correcting a model's violation, and hence turns the violation into an advantage.
  arXiv  Detail & Related papers  (2023-07-08T03:39:22Z)
• Truly Unordered Probabilistic Rule Sets for Multi-class Classification [0.0]
  We propose TURS, for Truly Unordered Rule Sets. We first formalise the problem of learning truly unordered rule sets. We then develop a two-phase algorithm that learns rule sets by carefully growing rules.
  arXiv  Detail & Related papers  (2022-06-17T14:34:35Z)
• Differentiable Rule Induction with Learned Relational Features [9.193818627108572]
  Rule Network (RRN) is a neural architecture that learns predicates that represent a linear relationship among attributes along with the rules that use them. On benchmark tasks we show that these predicates are simple enough to retain interpretability, yet improve prediction accuracy and provide sets of rules that are more concise compared to state of the art rule induction algorithms.
  arXiv  Detail & Related papers  (2022-01-17T16:46:50Z)
• Discovering Useful Compact Sets of Sequential Rules in a Long Sequence [57.684967309375274]
  COSSU is an algorithm to mine small and meaningful sets of sequential rules. We show that COSSU can successfully retrieve relevant sets of closed sequential rules from a long sequence.
  arXiv  Detail & Related papers  (2021-09-15T18:25:18Z)
• LNN-EL: A Neuro-Symbolic Approach to Short-text Entity Linking [62.634516517844496]
  We propose LNN-EL, a neuro-symbolic approach that combines the advantages of using interpretable rules with the performance of neural learning. Even though constrained to using rules, LNN-EL performs competitively against SotA black-box neural approaches.
  arXiv  Detail & Related papers  (2021-06-17T20:22:45Z)
• Better Short than Greedy: Interpretable Models through Optimal Rule Boosting [10.938624307941197]
  Rule ensembles are designed to provide a useful trade-off between predictive accuracy and model interpretability. We present a novel approach aiming to fit rule ensembles of maximal predictive power for a given ensemble size.
  arXiv  Detail & Related papers  (2021-01-21T01:03:48Z)
• Rewriting a Deep Generative Model [56.91974064348137]
  We introduce a new problem setting: manipulation of specific rules encoded by a deep generative model. We propose a formulation in which the desired rule is changed by manipulating a layer of a deep network as a linear associative memory. We present a user interface to enable users to interactively change the rules of a generative model to achieve desired effects.
  arXiv  Detail & Related papers  (2020-07-30T17:58:16Z)
• Diverse Rule Sets [20.170305081348328]
  Rule-based systems are experiencing a renaissance owing to their intuitive if-then representation. We propose a novel approach of inferring diverse rule sets, by optimizing small overlap among decision rules. We then devise an efficient randomized algorithm, which samples rules that are highly discriminative and have small overlap.
  arXiv  Detail & Related papers  (2020-06-17T14:15:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.