A New Baseline Assumption of Integated Gradients Based on Shaply value

• URL: http://arxiv.org/abs/2310.04821v3
• Date: Mon, 20 May 2024 13:44:46 GMT
• Title: A New Baseline Assumption of Integated Gradients Based on Shaply value
• Authors: Shuyang Liu, Zixuan Chen, Ge Shi, Ji Wang, Changjie Fan, Yu Xiong, Runze Wu Yujing Hu, Ze Ji, Yang Gao,
• Abstract summary: Integrated Gradients (IG) is a technique for mapping predictions back to the input features of deep neural networks (DNNs) We argue that the standard approach of utilizing a single baseline is frequently inadequate, prompting the need for multiple baselines. We develop a new baseline method called Shapley Integrated Gradients ( SIG), which uses proportional sampling to mirror the Shapley Value process.
• Score: 31.2051113305947
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Efforts to decode deep neural networks (DNNs) often involve mapping their predictions back to the input features. Among these methods, Integrated Gradients (IG) has emerged as a significant technique. The selection of appropriate baselines in IG is crucial for crafting meaningful and unbiased explanations of model predictions in diverse settings. The standard approach of utilizing a single baseline, however, is frequently inadequate, prompting the need for multiple baselines. Leveraging the natural link between IG and the Aumann-Shapley Value, we provide a novel outlook on baseline design. Theoretically, we demonstrate that under certain assumptions, a collection of baselines aligns with the coalitions described by the Shapley Value. Building on this insight, we develop a new baseline method called Shapley Integrated Gradients (SIG), which uses proportional sampling to mirror the Shapley Value computation process. Simulations conducted in GridWorld validate that SIG effectively emulates the distribution of Shapley Values. Moreover, empirical tests on various image processing tasks show that SIG surpasses traditional IG baseline methods by offering more precise estimates of feature contributions, providing consistent explanations across different applications, and ensuring adaptability to diverse data types with negligible additional computational demand.

Related papers

• Shapley-Guided Utility Learning for Effective Graph Inference Data Valuation [6.542796128290513]
  We propose Shapley-Guided Utility Learning (SGUL), a novel framework for graph inference data valuation. SGUL combines transferable data-specific and modelspecific features to approximate test accuracy without relying on ground truth labels. We show that SGUL consistently outperforms existing baselines in both inductive and transductive settings.
  arXiv  Detail & Related papers  (2025-03-23T20:35:03Z)
• LRP-QViT: Mixed-Precision Vision Transformer Quantization via Layer-wise Relevance Propagation [0.0]
  We introduce LRP-QViT, an explainability-based method for assigning mixed-precision bit allocations to different layers based on their importance during classification. Our experimental findings demonstrate that both our fixed-bit and mixed-bit post-training quantization methods surpass existing models in the context of 4-bit and 6-bit quantization.
  arXiv  Detail & Related papers  (2024-01-20T14:53:19Z)
• Shapley Values-enabled Progressive Pseudo Bag Augmentation for Whole Slide Image Classification [6.705260410604528]
  In computational pathology, whole-slide image (WSI) classification presents a formidable challenge due to its gigapixel resolution and limited fine-grained annotations. Multiple-instance learning (MIL) offers a weakly supervised solution, yet refining instance-level information from bag-level labels remains challenging. We propose a new approach inspired by cooperative game theory: employing Shapley values to assess each instance's contribution, thereby improving IIS estimation.
  arXiv  Detail & Related papers  (2023-12-09T07:35:09Z)
• Fast Shapley Value Estimation: A Unified Approach [71.92014859992263]
  We propose a straightforward and efficient Shapley estimator, SimSHAP, by eliminating redundant techniques. In our analysis of existing approaches, we observe that estimators can be unified as a linear transformation of randomly summed values from feature subsets. Our experiments validate the effectiveness of our SimSHAP, which significantly accelerates the computation of accurate Shapley values.
  arXiv  Detail & Related papers  (2023-11-02T06:09:24Z)
• RGM: A Robust Generalizable Matching Model [49.60975442871967]
  We propose a deep model for sparse and dense matching, termed RGM (Robust Generalist Matching) To narrow the gap between synthetic training samples and real-world scenarios, we build a new, large-scale dataset with sparse correspondence ground truth. We are able to mix up various dense and sparse matching datasets, significantly improving the training diversity.
  arXiv  Detail & Related papers  (2023-10-18T07:30:08Z)
• Beyond Deep Ensembles: A Large-Scale Evaluation of Bayesian Deep Learning under Distribution Shift [19.945634052291542]
  We evaluate modern BDL algorithms on real-world datasets from the WILDS collection containing challenging classification and regression tasks. We compare the algorithms on a wide range of large, convolutional and transformer-based neural network architectures. We provide the first systematic evaluation of BDL for fine-tuning large pre-trained models.
  arXiv  Detail & Related papers  (2023-06-21T14:36:03Z)
• Precision-Recall Divergence Optimization for Generative Modeling with GANs and Normalizing Flows [54.050498411883495]
  We develop a novel training method for generative models, such as Generative Adversarial Networks and Normalizing Flows. We show that achieving a specified precision-recall trade-off corresponds to minimizing a unique $f$-divergence from a family we call the textitPR-divergences. Our approach improves the performance of existing state-of-the-art models like BigGAN in terms of either precision or recall when tested on datasets such as ImageNet.
  arXiv  Detail & Related papers  (2023-05-30T10:07:17Z)
• Interpretation of Neural Networks is Susceptible to Universal Adversarial Perturbations [9.054540533394926]
  We show the existence of a Universal Perturbation for Interpretation (UPI) for standard image datasets. We propose a gradient-based optimization method as well as a principal component analysis (PCA)-based approach to compute a UPI which can effectively alter a neural network's gradient-based interpretation on different samples.
  arXiv  Detail & Related papers  (2022-11-30T15:55:40Z)
• Incremental Ensemble Gaussian Processes [53.3291389385672]
  We propose an incremental ensemble (IE-) GP framework, where an EGP meta-learner employs an it ensemble of GP learners, each having a unique kernel belonging to a prescribed kernel dictionary. With each GP expert leveraging the random feature-based approximation to perform online prediction and model update with it scalability, the EGP meta-learner capitalizes on data-adaptive weights to synthesize the per-expert predictions. The novel IE-GP is generalized to accommodate time-varying functions by modeling structured dynamics at the EGP meta-learner and within each GP learner.
  arXiv  Detail & Related papers  (2021-10-13T15:11:25Z)
• Neural BRDF Representation and Importance Sampling [79.84316447473873]
  We present a compact neural network-based representation of reflectance BRDF data. We encode BRDFs as lightweight networks, and propose a training scheme with adaptive angular sampling. We evaluate encoding results on isotropic and anisotropic BRDFs from multiple real-world datasets.
  arXiv  Detail & Related papers  (2021-02-11T12:00:24Z)
• Making Affine Correspondences Work in Camera Geometry Computation [62.7633180470428]
  Local features provide region-to-region rather than point-to-point correspondences. We propose guidelines for effective use of region-to-region matches in the course of a full model estimation pipeline. Experiments show that affine solvers can achieve accuracy comparable to point-based solvers at faster run-times.
  arXiv  Detail & Related papers  (2020-07-20T12:07:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.