Learning Interpretable Differentiable Logic Networks

• URL: http://arxiv.org/abs/2407.04168v1
• Date: Thu, 4 Jul 2024 21:58:26 GMT
• Title: Learning Interpretable Differentiable Logic Networks
• Authors: Chang Yue, Niraj K. Jha,
• Abstract summary: We introduce a novel method for learning interpretable differentiable logic networks (DLNs) We train these networks by softening and differentiating their discrete components, through binarization of inputs, binary logic operations, and connections between neurons. Experimental results on twenty classification tasks indicate that differentiable logic networks can achieve accuracies comparable to or exceeding that of traditional NNs.
• Score: 3.8064485653035987
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The ubiquity of neural networks (NNs) in real-world applications, from healthcare to natural language processing, underscores their immense utility in capturing complex relationships within high-dimensional data. However, NNs come with notable disadvantages, such as their "black-box" nature, which hampers interpretability, as well as their tendency to overfit the training data. We introduce a novel method for learning interpretable differentiable logic networks (DLNs) that are architectures that employ multiple layers of binary logic operators. We train these networks by softening and differentiating their discrete components, e.g., through binarization of inputs, binary logic operations, and connections between neurons. This approach enables the use of gradient-based learning methods. Experimental results on twenty classification tasks indicate that differentiable logic networks can achieve accuracies comparable to or exceeding that of traditional NNs. Equally importantly, these networks offer the advantage of interpretability. Moreover, their relatively simple structure results in the number of logic gate-level operations during inference being up to a thousand times smaller than NNs, making them suitable for deployment on edge devices.

Related papers

• Neural Reasoning Networks: Efficient Interpretable Neural Networks With Automatic Textual Explanations [45.974930902038494]
  We propose a novel neuro-symbolic architecture, Neural Reasoning Networks (NRN), that is scalable and generates logically textual explanations for its predictions. A training algorithm (R-NRN) learns the weights of the network as usual using descent optimization with backprop, but also learns the network structure itself using a bandit-based optimization. R-NRN explanations are shorter than the compared approaches while producing more accurate feature importance scores.
  arXiv  Detail & Related papers  (2024-10-10T14:27:12Z)
• GINN-KAN: Interpretability pipelining with applications in Physics Informed Neural Networks [5.2969467015867915]
  We introduce the concept of interpretability pipelineing, to incorporate multiple interpretability techniques to outperform each individual technique. We evaluate two recent models selected for their potential to incorporate interpretability into standard neural network architectures. We introduce a novel interpretable neural network GINN-KAN that synthesizes the advantages of both models.
  arXiv  Detail & Related papers  (2024-08-27T04:57:53Z)
• Neural networks trained with SGD learn distributions of increasing complexity [78.30235086565388]
  We show that neural networks trained using gradient descent initially classify their inputs using lower-order input statistics. We then exploit higher-order statistics only later during training. We discuss the relation of DSB to other simplicity biases and consider its implications for the principle of universality in learning.
  arXiv  Detail & Related papers  (2022-11-21T15:27:22Z)
• Interpretable part-whole hierarchies and conceptual-semantic relationships in neural networks [4.153804257347222]
  We present Agglomerator, a framework capable of providing a representation of part-whole hierarchies from visual cues. We evaluate our method on common datasets, such as SmallNORB, MNIST, FashionMNIST, CIFAR-10, and CIFAR-100.
  arXiv  Detail & Related papers  (2022-03-07T10:56:13Z)
• Neuro-Symbolic Inductive Logic Programming with Logical Neural Networks [65.23508422635862]
  We propose learning rules with the recently proposed logical neural networks (LNN) Compared to others, LNNs offer strong connection to classical Boolean logic. Our experiments on standard benchmarking tasks confirm that LNN rules are highly interpretable.
  arXiv  Detail & Related papers  (2021-12-06T19:38:30Z)
• Distributed Learning for Time-varying Networks: A Scalable Design [13.657740129012804]
  We propose a distributed learning framework based on a scalable deep neural network (DNN) design. By exploiting the permutation equivalence and invariance properties of the learning tasks, the DNNs with different scales for different clients can be built up. Model aggregation can also be conducted based on these two sub-matrices to improve the learning convergence and performance.
  arXiv  Detail & Related papers  (2021-07-31T12:44:28Z)
• FF-NSL: Feed-Forward Neural-Symbolic Learner [70.978007919101]
  This paper introduces a neural-symbolic learning framework, called Feed-Forward Neural-Symbolic Learner (FF-NSL) FF-NSL integrates state-of-the-art ILP systems based on the Answer Set semantics, with neural networks, in order to learn interpretable hypotheses from labelled unstructured data.
  arXiv  Detail & Related papers  (2021-06-24T15:38:34Z)
• Reinforcement Learning with External Knowledge by using Logical Neural Networks [67.46162586940905]
  A recent neuro-symbolic framework called the Logical Neural Networks (LNNs) can simultaneously provide key-properties of both neural networks and symbolic logic. We propose an integrated method that enables model-free reinforcement learning from external knowledge sources.
  arXiv  Detail & Related papers  (2021-03-03T12:34:59Z)
• Learning Connectivity of Neural Networks from a Topological Perspective [80.35103711638548]
  We propose a topological perspective to represent a network into a complete graph for analysis. By assigning learnable parameters to the edges which reflect the magnitude of connections, the learning process can be performed in a differentiable manner. This learning process is compatible with existing networks and owns adaptability to larger search spaces and different tasks.
  arXiv  Detail & Related papers  (2020-08-19T04:53:31Z)
• Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks [80.15411508088522]
  Spiking neural networks (SNNs) have shown advantages over traditional artificial neural networks (ANNs) for low latency and high computational efficiency. We propose a novel ANN-to-SNN conversion and layer-wise learning framework for rapid and efficient pattern recognition.
  arXiv  Detail & Related papers  (2020-07-02T15:38:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.