Rule Based Learning with Dynamic (Graph) Neural Networks

• URL: http://arxiv.org/abs/2406.09954v1
• Date: Fri, 14 Jun 2024 12:01:18 GMT
• Title: Rule Based Learning with Dynamic (Graph) Neural Networks
• Authors: Florian Seiffarth,
• Abstract summary: We present rule based graph neural networks (RuleGNNs) that overcome some limitations of ordinary graph neural networks. Our experiments show that the predictive performance of RuleGNNs is comparable to state-of-the-art graph classifiers. We introduce new synthetic benchmark graph datasets to show how to integrate expert knowledge into RuleGNNs.
• Score: 0.8158530638728501
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A common problem of classical neural network architectures is that additional information or expert knowledge cannot be naturally integrated into the learning process. To overcome this limitation, we propose a two-step approach consisting of (1) generating rule functions from knowledge and (2) using these rules to define rule based layers -- a new type of dynamic neural network layer. The focus of this work is on the second step, i.e., rule based layers that are designed to dynamically arrange learnable parameters in the weight matrices and bias vectors depending on the input samples. Indeed, we prove that our approach generalizes classical feed-forward layers such as fully connected and convolutional layers by choosing appropriate rules. As a concrete application we present rule based graph neural networks (RuleGNNs) that overcome some limitations of ordinary graph neural networks. Our experiments show that the predictive performance of RuleGNNs is comparable to state-of-the-art graph classifiers using simple rules based on Weisfeiler-Leman labeling and pattern counting. Moreover, we introduce new synthetic benchmark graph datasets to show how to integrate expert knowledge into RuleGNNs making them more powerful than ordinary graph neural networks.

Related papers

• Neural Symbolic Logical Rule Learner for Interpretable Learning [1.9526476410335776]
  Rule-based neural networks stand out for enabling interpretable classification by learning logical rules for both prediction and interpretation. We introduce the Normal Form Rule Learner (NFRL) algorithm, leveraging a selective discrete neural network, to learn rules in both Conjunctive Normal Form (CNF) and Disjunctive Normal Form (DNF) Through extensive experiments on 11 datasets, NFRL demonstrates superior classification performance, quality of learned rules, efficiency and interpretability compared to 12 state-of-the-art alternatives.
  arXiv  Detail & Related papers  (2024-08-21T18:09:12Z)
• Neuro-symbolic Rule Learning in Real-world Classification Tasks [75.0907310059298]
  We extend pix2rule's neural DNF module to support rule learning in real-world multi-class and multi-label classification tasks. We propose a novel extended model called neural DNF-EO (Exactly One) which enforces mutual exclusivity in multi-class classification.
  arXiv  Detail & Related papers  (2023-03-29T13:27:14Z)
• Knowledge Enhanced Neural Networks for relational domains [83.9217787335878]
  We focus on a specific method, KENN, a Neural-Symbolic architecture that injects prior logical knowledge into a neural network. In this paper, we propose an extension of KENN for relational data.
  arXiv  Detail & Related papers  (2022-05-31T13:00:34Z)
• On the Interpretability of Regularisation for Neural Networks Through Model Gradient Similarity [0.0]
  Model Gradient Similarity (MGS) serves as a metric of regularisation. MGS provides the basis for a new regularisation scheme which exhibits excellent performance.
  arXiv  Detail & Related papers  (2022-05-25T10:38:33Z)
• Meta-Learning Bidirectional Update Rules [14.397000142362337]
  We introduce a new type of generalized neural network where neurons and synapses maintain multiple states. We show that classical-based backpropagation in neural networks can be seen as a special case of a two-state network where one state is used for activations and another for gradient, with update rules derived from the chain rule.
  arXiv  Detail & Related papers  (2021-04-10T00:56:35Z)
• Learning Accurate and Interpretable Decision Rule Sets from Neural Networks [5.280792199222362]
  This paper proposes a new paradigm for learning a set of independent logical rules in disjunctive normal form as an interpretable model for classification. We consider the problem of learning an interpretable decision rule set as training a neural network in a specific, yet very simple two-layer architecture.
  arXiv  Detail & Related papers  (2021-03-04T04:10:19Z)
• Binary Graph Neural Networks [69.51765073772226]
  Graph Neural Networks (GNNs) have emerged as a powerful and flexible framework for representation learning on irregular data. In this paper, we present and evaluate different strategies for the binarization of graph neural networks. We show that through careful design of the models, and control of the training process, binary graph neural networks can be trained at only a moderate cost in accuracy on challenging benchmarks.
  arXiv  Detail & Related papers  (2020-12-31T18:48:58Z)
• Connecting Weighted Automata, Tensor Networks and Recurrent Neural Networks through Spectral Learning [58.14930566993063]
  We present connections between three models used in different research fields: weighted finite automata(WFA) from formal languages and linguistics, recurrent neural networks used in machine learning, and tensor networks. We introduce the first provable learning algorithm for linear 2-RNN defined over sequences of continuous vectors input.
  arXiv  Detail & Related papers  (2020-10-19T15:28:00Z)
• Data-Driven Learning of Geometric Scattering Networks [74.3283600072357]
  We propose a new graph neural network (GNN) module based on relaxations of recently proposed geometric scattering transforms. Our learnable geometric scattering (LEGS) module enables adaptive tuning of the wavelets to encourage band-pass features to emerge in learned representations.
  arXiv  Detail & Related papers  (2020-10-06T01:20:27Z)
• Binarized Graph Neural Network [65.20589262811677]
  We develop a binarized graph neural network to learn the binary representations of the nodes with binary network parameters. Our proposed method can be seamlessly integrated into the existing GNN-based embedding approaches. Experiments indicate that the proposed binarized graph neural network, namely BGN, is orders of magnitude more efficient in terms of both time and space.
  arXiv  Detail & Related papers  (2020-04-19T09:43:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.