Generalizing Backpropagation for Gradient-Based Interpretability
- URL: http://arxiv.org/abs/2307.03056v1
- Date: Thu, 6 Jul 2023 15:19:53 GMT
- Title: Generalizing Backpropagation for Gradient-Based Interpretability
- Authors: Kevin Du, Lucas Torroba Hennigen, Niklas Stoehr, Alexander Warstadt,
Ryan Cotterell
- Abstract summary: We show that the gradient of a model is a special case of a more general formulation using semirings.
This observation allows us to generalize the backpropagation algorithm to efficiently compute other interpretable statistics.
- Score: 103.2998254573497
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Many popular feature-attribution methods for interpreting deep neural
networks rely on computing the gradients of a model's output with respect to
its inputs. While these methods can indicate which input features may be
important for the model's prediction, they reveal little about the inner
workings of the model itself. In this paper, we observe that the gradient
computation of a model is a special case of a more general formulation using
semirings. This observation allows us to generalize the backpropagation
algorithm to efficiently compute other interpretable statistics about the
gradient graph of a neural network, such as the highest-weighted path and
entropy. We implement this generalized algorithm, evaluate it on synthetic
datasets to better understand the statistics it computes, and apply it to study
BERT's behavior on the subject-verb number agreement task (SVA). With this
method, we (a) validate that the amount of gradient flow through a component of
a model reflects its importance to a prediction and (b) for SVA, identify which
pathways of the self-attention mechanism are most important.
Related papers
- Supervised Score-Based Modeling by Gradient Boosting [49.556736252628745]
We propose a Supervised Score-based Model (SSM) which can be viewed as a gradient boosting algorithm combining score matching.
We provide a theoretical analysis of learning and sampling for SSM to balance inference time and prediction accuracy.
Our model outperforms existing models in both accuracy and inference time.
arXiv Detail & Related papers (2024-11-02T07:06:53Z) - Sub-graph Based Diffusion Model for Link Prediction [43.15741675617231]
Denoising Diffusion Probabilistic Models (DDPMs) represent a contemporary class of generative models with exceptional qualities.
We build a novel generative model for link prediction using a dedicated design to decompose the likelihood estimation process via the Bayesian formula.
Our proposed method presents numerous advantages: (1) transferability across datasets without retraining, (2) promising generalization on limited training data, and (3) robustness against graph adversarial attacks.
arXiv Detail & Related papers (2024-09-13T02:23:55Z) - Estimating Causal Effects from Learned Causal Networks [56.14597641617531]
We propose an alternative paradigm for answering causal-effect queries over discrete observable variables.
We learn the causal Bayesian network and its confounding latent variables directly from the observational data.
We show that this emphmodel completion learning approach can be more effective than estimand approaches.
arXiv Detail & Related papers (2024-08-26T08:39:09Z) - Correcting Model Bias with Sparse Implicit Processes [0.9187159782788579]
We show that Sparse Implicit Processes (SIP) is capable of correcting model bias when the data generating mechanism differs strongly from the one implied by the model.
We use synthetic datasets to show that SIP is capable of providing predictive distributions that reflect the data better than the exact predictions of the initial, but wrongly assumed model.
arXiv Detail & Related papers (2022-07-21T18:00:01Z) - Scalable computation of prediction intervals for neural networks via
matrix sketching [79.44177623781043]
Existing algorithms for uncertainty estimation require modifying the model architecture and training procedure.
This work proposes a new algorithm that can be applied to a given trained neural network and produces approximate prediction intervals.
arXiv Detail & Related papers (2022-05-06T13:18:31Z) - Bayesian Graph Contrastive Learning [55.36652660268726]
We propose a novel perspective of graph contrastive learning methods showing random augmentations leads to encoders.
Our proposed method represents each node by a distribution in the latent space in contrast to existing techniques which embed each node to a deterministic vector.
We show a considerable improvement in performance compared to existing state-of-the-art methods on several benchmark datasets.
arXiv Detail & Related papers (2021-12-15T01:45:32Z) - Integrated Grad-CAM: Sensitivity-Aware Visual Explanation of Deep
Convolutional Networks via Integrated Gradient-Based Scoring [26.434705114982584]
Grad-CAM is a popular solution that provides such a visualization by combining the activation maps obtained from the model.
We introduce a solution to tackle this problem by computing the path integral of the gradient-based terms in Grad-CAM.
We conduct a thorough analysis to demonstrate the improvement achieved by our method in measuring the importance of the extracted representations for the CNN's predictions.
arXiv Detail & Related papers (2021-02-15T19:21:46Z) - Goal-directed Generation of Discrete Structures with Conditional
Generative Models [85.51463588099556]
We introduce a novel approach to directly optimize a reinforcement learning objective, maximizing an expected reward.
We test our methodology on two tasks: generating molecules with user-defined properties and identifying short python expressions which evaluate to a given target value.
arXiv Detail & Related papers (2020-10-05T20:03:13Z) - SODEN: A Scalable Continuous-Time Survival Model through Ordinary
Differential Equation Networks [14.564168076456822]
We propose a flexible model for survival analysis using neural networks along with scalable optimization algorithms.
We demonstrate the effectiveness of the proposed method in comparison to existing state-of-the-art deep learning survival analysis models.
arXiv Detail & Related papers (2020-08-19T19:11:25Z) - Relative gradient optimization of the Jacobian term in unsupervised deep
learning [9.385902422987677]
Learning expressive probabilistic models correctly describing the data is a ubiquitous problem in machine learning.
Deep density models have been widely used for this task, but their maximum likelihood based training requires estimating the log-determinant of the Jacobian.
We propose a new approach for exact training of such neural networks.
arXiv Detail & Related papers (2020-06-26T16:41:08Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.