Sum-of-Parts: Faithful Attributions for Groups of Features

• URL: http://arxiv.org/abs/2310.16316v2
• Date: Wed, 02 Oct 2024 23:37:28 GMT
• Title: Sum-of-Parts: Faithful Attributions for Groups of Features
• Authors: Weiqiu You, Helen Qu, Marco Gatti, Bhuvnesh Jain, Eric Wong,
• Abstract summary: Sum-of-Parts ( SOP) is a framework that transforms any differentiable model into a self-explaining model whose predictions can be attributed to groups of features. SOP achieves highest performance while also scoring high with respect to faithfulness metrics on ImageNet and CosmoGrid. We validate the usefulness of the groups learned by SOP through their high purity, strong human distinction ability, and practical utility in scientific discovery.
• Score: 8.68707471649733
• License:
• Abstract: Feature attributions explain machine learning predictions by assigning importance scores to input features. While faithful attributions accurately reflect feature contributions to the model's prediction, unfaithful ones can lead to misleading interpretations, making them unreliable in high-stake domains. The challenge of unfaithfulness of post-hoc attributions led to the development of self-explaining models. However, self-explaining models often trade-off performance for interpretability. In this work, we develop Sum-of-Parts (SOP), a new framework that transforms any differentiable model into a self-explaining model whose predictions can be attributed to groups of features. The SOP framework leverages pretrained deep learning models with custom attention modules to learn useful feature groups end-to-end without direct supervision. With these capabilities, SOP achieves highest performance while also scoring high with respect to faithfulness metrics on both ImageNet and CosmoGrid. We validate the usefulness of the groups learned by SOP through their high purity, strong human distinction ability, and practical utility in scientific discovery. In a case study, we show how SOP assists cosmologists in uncovering new insights about galaxy formation.

Related papers

• Towards Scalable and Versatile Weight Space Learning [51.78426981947659]
  This paper introduces the SANE approach to weight-space learning. Our method extends the idea of hyper-representations towards sequential processing of subsets of neural network weights.
  arXiv  Detail & Related papers  (2024-06-14T13:12:07Z)
• Discover, Explanation, Improvement: An Automatic Slice Detection Framework for Natural Language Processing [72.14557106085284]
  slice detection models (SDM) automatically identify underperforming groups of datapoints. This paper proposes a benchmark named "Discover, Explain, improve (DEIM)" for classification NLP tasks. Our evaluation shows that Edisa can accurately select error-prone datapoints with informative semantic features.
  arXiv  Detail & Related papers  (2022-11-08T19:00:00Z)
• Equivariant Transduction through Invariant Alignment [71.45263447328374]
  We introduce a novel group-equivariant architecture that incorporates a group-in hard alignment mechanism. We find that our network's structure allows it to develop stronger equivariant properties than existing group-equivariant approaches. We additionally find that it outperforms previous group-equivariant networks empirically on the SCAN task.
  arXiv  Detail & Related papers  (2022-09-22T11:19:45Z)
• Batch-Ensemble Stochastic Neural Networks for Out-of-Distribution Detection [55.028065567756066]
  Out-of-distribution (OOD) detection has recently received much attention from the machine learning community due to its importance in deploying machine learning models in real-world applications. In this paper we propose an uncertainty quantification approach by modelling the distribution of features. We incorporate an efficient ensemble mechanism, namely batch-ensemble, to construct the batch-ensemble neural networks (BE-SNNs) and overcome the feature collapse problem. We show that BE-SNNs yield superior performance on several OOD benchmarks, such as the Two-Moons dataset, the FashionMNIST vs MNIST dataset, FashionM
  arXiv  Detail & Related papers  (2022-06-26T16:00:22Z)
• Self-Ensembling GAN for Cross-Domain Semantic Segmentation [107.27377745720243]
  This paper proposes a self-ensembling generative adversarial network (SE-GAN) exploiting cross-domain data for semantic segmentation. In SE-GAN, a teacher network and a student network constitute a self-ensembling model for generating semantic segmentation maps, which together with a discriminator, forms a GAN. Despite its simplicity, we find SE-GAN can significantly boost the performance of adversarial training and enhance the stability of the model.
  arXiv  Detail & Related papers  (2021-12-15T09:50:25Z)
• Subgroup Generalization and Fairness of Graph Neural Networks [12.88476464580968]
  We present a novel PAC-Bayesian analysis for GNNs under a non-IID semi-supervised learning setup. We further study an accuracy-(dis)parity-style (un)fairness of GNNs from a theoretical perspective.
  arXiv  Detail & Related papers  (2021-06-29T16:13:41Z)
• Group Equivariant Neural Architecture Search via Group Decomposition and Reinforcement Learning [17.291131923335918]
  We prove a new group-theoretic result in the context of equivariant neural networks. We also design an algorithm to construct equivariant networks that significantly improves computational complexity. We use deep Q-learning to search for group equivariant networks that maximize performance.
  arXiv  Detail & Related papers  (2021-04-10T19:37:25Z)
• Contextual Classification Using Self-Supervised Auxiliary Models for Deep Neural Networks [6.585049648605185]
  We introduce the notion of Self-Supervised Autogenous Learning (SSAL) models. A SSAL objective is realized through one or more additional targets that are derived from the original supervised classification task. We show that SSAL models consistently outperform the state-of-the-art while also providing structured predictions that are more interpretable.
  arXiv  Detail & Related papers  (2021-01-07T18:41:16Z)
• Ensembles of Spiking Neural Networks [0.3007949058551534]
  This paper demonstrates how to construct ensembles of spiking neural networks producing state-of-the-art results. We achieve classification accuracies of 98.71%, 100.0%, and 99.09%, on the MNIST, NMNIST and DVS Gesture datasets respectively. We formalize spiking neural networks as GLM predictors, identifying a suitable representation for their target domain.
  arXiv  Detail & Related papers  (2020-10-15T17:45:18Z)
• Rethinking Generalization of Neural Models: A Named Entity Recognition Case Study [81.11161697133095]
  We take the NER task as a testbed to analyze the generalization behavior of existing models from different perspectives. Experiments with in-depth analyses diagnose the bottleneck of existing neural NER models. As a by-product of this paper, we have open-sourced a project that involves a comprehensive summary of recent NER papers.
  arXiv  Detail & Related papers  (2020-01-12T04:33:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.