Improving Interpretability and Accuracy in Neuro-Symbolic Rule Extraction Using Class-Specific Sparse Filters

• URL: http://arxiv.org/abs/2501.16677v1
• Date: Tue, 28 Jan 2025 03:22:23 GMT
• Title: Improving Interpretability and Accuracy in Neuro-Symbolic Rule Extraction Using Class-Specific Sparse Filters
• Authors: Parth Padalkar, Jaeseong Lee, Shiyi Wei, Gopal Gupta,
• Abstract summary: We propose a novel sparsity loss function that enables class-specific filter binarization during CNN training. We set a new benchmark, achieving a 9% improvement in accuracy and a 53% reduction in rule-set size on average. This highlights the significant potential of interpretable neuro-symbolic models as viable alternatives to black-box CNNs.
• Score: 5.1690347717311855
• License:
• Abstract: There has been significant focus on creating neuro-symbolic models for interpretable image classification using Convolutional Neural Networks (CNNs). These methods aim to replace the CNN with a neuro-symbolic model consisting of the CNN, which is used as a feature extractor, and an interpretable rule-set extracted from the CNN itself. While these approaches provide interpretability through the extracted rule-set, they often compromise accuracy compared to the original CNN model. In this paper, we identify the root cause of this accuracy loss as the post-training binarization of filter activations to extract the rule-set. To address this, we propose a novel sparsity loss function that enables class-specific filter binarization during CNN training, thus minimizing information loss when extracting the rule-set. We evaluate several training strategies with our novel sparsity loss, analyzing their effectiveness and providing guidance on their appropriate use. Notably, we set a new benchmark, achieving a 9% improvement in accuracy and a 53% reduction in rule-set size on average, compared to the previous SOTA, while coming within 3% of the original CNN's accuracy. This highlights the significant potential of interpretable neuro-symbolic models as viable alternatives to black-box CNNs.

Related papers

• On the rates of convergence for learning with convolutional neural networks [9.772773527230134]
  We study approximation and learning capacities of convolutional neural networks (CNNs) with one-side zero-padding and multiple channels. We derive convergence rates for estimators based on CNNs in many learning problems. It is also shown that the obtained rates for classification are minimax optimal in some common settings.
  arXiv  Detail & Related papers  (2024-03-25T06:42:02Z)
• Improving the Accuracy and Robustness of CNNs Using a Deep CCA Neural Data Regularizer [2.026424957803652]
  As convolutional neural networks (CNNs) become more accurate at object recognition, their representations become more similar to the primate visual system. Previous attempts to address this question showed very modest gains in accuracy, owing in part to limitations of the regularization method. We develop a new neural data regularizer for CNNs that uses Deep Correlation Analysis (DCCA) to optimize the resemblance of the CNN's image representations to that of the monkey visual cortex.
  arXiv  Detail & Related papers  (2022-09-06T15:40:39Z)
• Batch Normalization Tells You Which Filter is Important [49.903610684578716]
  We propose a simple yet effective filter pruning method by evaluating the importance of each filter based on the BN parameters of pre-trained CNNs. The experimental results on CIFAR-10 and ImageNet demonstrate that the proposed method can achieve outstanding performance.
  arXiv  Detail & Related papers  (2021-12-02T12:04:59Z)
• Neural Architecture Dilation for Adversarial Robustness [56.18555072877193]
  A shortcoming of convolutional neural networks is that they are vulnerable to adversarial attacks. This paper aims to improve the adversarial robustness of the backbone CNNs that have a satisfactory accuracy. Under a minimal computational overhead, a dilation architecture is expected to be friendly with the standard performance of the backbone CNN.
  arXiv  Detail & Related papers  (2021-08-16T03:58:00Z)
• Fairness via Representation Neutralization [60.90373932844308]
  We propose a new mitigation technique, namely, Representation Neutralization for Fairness (RNF) RNF achieves that fairness by debiasing only the task-specific classification head of DNN models. Experimental results over several benchmark datasets demonstrate our RNF framework to effectively reduce discrimination of DNN models.
  arXiv  Detail & Related papers  (2021-06-23T22:26:29Z)
• Examining and Mitigating Kernel Saturation in Convolutional Neural Networks using Negative Images [0.8594140167290097]
  We analyze the effect of convolutional kernel saturation in CNNs. We propose a simple data augmentation technique to mitigate saturation and increase classification accuracy, by supplementing negative images to the training dataset. Our results show that CNNs are indeed susceptible to convolutional kernel saturation and that supplementing negative images to the training dataset can offer a statistically significant increase in classification accuracies.
  arXiv  Detail & Related papers  (2021-05-10T06:06:49Z)
• BreakingBED -- Breaking Binary and Efficient Deep Neural Networks by Adversarial Attacks [65.2021953284622]
  We study robustness of CNNs against white-box and black-box adversarial attacks. Results are shown for distilled CNNs, agent-based state-of-the-art pruned models, and binarized neural networks.
  arXiv  Detail & Related papers  (2021-03-14T20:43:19Z)
• CNNPruner: Pruning Convolutional Neural Networks with Visual Analytics [13.38218193857018]
  Convolutional neural networks (CNNs) have demonstrated extraordinarily good performance in many computer vision tasks. CNNPruner allows users to interactively create pruning plans according to a desired goal on model size or accuracy.
  arXiv  Detail & Related papers  (2020-09-08T02:08:20Z)
• RIFLE: Backpropagation in Depth for Deep Transfer Learning through Re-Initializing the Fully-connected LayEr [60.07531696857743]
  Fine-tuning the deep convolution neural network(CNN) using a pre-trained model helps transfer knowledge learned from larger datasets to the target task. We propose RIFLE - a strategy that deepens backpropagation in transfer learning settings. RIFLE brings meaningful updates to the weights of deep CNN layers and improves low-level feature learning.
  arXiv  Detail & Related papers  (2020-07-07T11:27:43Z)
• Approximation and Non-parametric Estimation of ResNet-type Convolutional Neural Networks [52.972605601174955]
  We show a ResNet-type CNN can attain the minimax optimal error rates in important function classes. We derive approximation and estimation error rates of the aformentioned type of CNNs for the Barron and H"older classes.
  arXiv  Detail & Related papers  (2019-03-24T19:42:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.