Using the Path of Least Resistance to Explain Deep Networks

• URL: http://arxiv.org/abs/2502.12108v1
• Date: Mon, 17 Feb 2025 18:29:24 GMT
• Title: Using the Path of Least Resistance to Explain Deep Networks
• Authors: Sina Salek, Joseph Enguehard,
• Abstract summary: Integrated Gradients (IG) is a widely used axiomatic path-based attribution method. We show that straight paths can lead to flawed attributions. We propose Geodesic Integrated Gradients (GIG) as an alternative.
• Score: 5.614094161229764
• License:
• Abstract: Integrated Gradients (IG), a widely used axiomatic path-based attribution method, assigns importance scores to input features by integrating model gradients along a straight path from a baseline to the input. While effective in some cases, we show that straight paths can lead to flawed attributions. In this paper, we identify the cause of these misattributions and propose an alternative approach that treats the input space as a Riemannian manifold, computing attributions by integrating gradients along geodesics. We call this method Geodesic Integrated Gradients (GIG). To approximate geodesic paths, we introduce two techniques: a k-Nearest Neighbours-based approach for smaller models and a Stochastic Variational Inference-based method for larger ones. Additionally, we propose a new axiom, Strong Completeness, extending the axioms satisfied by IG. We show that this property is desirable for attribution methods and that GIG is the only method that satisfies it. Through experiments on both synthetic and real-world data, we demonstrate that GIG outperforms existing explainability methods, including IG.

Related papers

• Graph-Sequential Alignment and Uniformity: Toward Enhanced Recommendation Systems [51.716704243764994]
  Our framework uses Graph Neural Network (GNN)-based and sequential recommenders as separate submodules while sharing a unified embedding space optimized jointly. Experiments on three real-world datasets demonstrate that the proposed method significantly outperforms using either approach alone.
  arXiv  Detail & Related papers  (2024-12-05T15:59:05Z)
• IG2: Integrated Gradient on Iterative Gradient Path for Feature Attribution [6.278326325782819]
  Iterative Gradient path Integrated Gradients (IG2) is a prominent path attribution method for deep neural networks. IG2 incorporates the counterfactual gradient iteratively into the integration path, generating a novel path (GradPath) and a novel baseline (GradCF) Experimental results on XAI benchmark, ImageNet, MNIST, TREC questions answering, wafer-map failure patterns, and CelebA face attributes validate that IG2 delivers superior feature attributions.
  arXiv  Detail & Related papers  (2024-06-16T08:48:03Z)
• Manifold Integrated Gradients: Riemannian Geometry for Feature Attribution [8.107199775668942]
  Integrated Gradients (IG) is a prevalent feature attribution method for black-box deep learning models. We address two predominant challenges associated with IG: the generation of noisy feature visualizations and the vulnerability to adversarial attributional attacks. Our approach involves an adaptation of path-based feature attribution, aligning the path of attribution more closely to the intrinsic geometry of the data manifold.
  arXiv  Detail & Related papers  (2024-05-16T04:13:17Z)
• Optimizing Solution-Samplers for Combinatorial Problems: The Landscape of Policy-Gradient Methods [52.0617030129699]
  We introduce a novel theoretical framework for analyzing the effectiveness of DeepMatching Networks and Reinforcement Learning methods. Our main contribution holds for a broad class of problems including Max-and Min-Cut, Max-$k$-Bipartite-Bi, Maximum-Weight-Bipartite-Bi, and Traveling Salesman Problem. As a byproduct of our analysis we introduce a novel regularization process over vanilla descent and provide theoretical and experimental evidence that it helps address vanishing-gradient issues and escape bad stationary points.
  arXiv  Detail & Related papers  (2023-10-08T23:39:38Z)
• A Heat Diffusion Perspective on Geodesic Preserving Dimensionality Reduction [66.21060114843202]
  We propose a more general heat kernel based manifold embedding method that we call heat geodesic embeddings. Results show that our method outperforms existing state of the art in preserving ground truth manifold distances. We also showcase our method on single cell RNA-sequencing datasets with both continuum and cluster structure.
  arXiv  Detail & Related papers  (2023-05-30T13:58:50Z)
• Interactive Segmentation as Gaussian Process Classification [58.44673380545409]
  Click-based interactive segmentation (IS) aims to extract the target objects under user interaction. Most of the current deep learning (DL)-based methods mainly follow the general pipelines of semantic segmentation. We propose to formulate the IS task as a Gaussian process (GP)-based pixel-wise binary classification model on each image.
  arXiv  Detail & Related papers  (2023-02-28T14:01:01Z)
• Geometrically Guided Integrated Gradients [0.3867363075280543]
  We introduce an interpretability method called "geometrically-guided integrated gradients" Our method explores the model's dynamic behavior from multiple scaled versions of the input and captures the best possible attribution for each input. We also propose a "model perturbation" sanity check to complement the traditionally used "model randomization" test.
  arXiv  Detail & Related papers  (2022-06-13T05:05:43Z)
• Guided Integrated Gradients: An Adaptive Path Method for Removing Noise [9.792727625917083]
  Integrated Gradients (IG) is a commonly used feature attribution method for deep neural networks. We show that one of the causes of the problem is the accumulation of noise along the IG path. We propose adapting the attribution path itself -- conditioning the path not just on the image but also on the model being explained.
  arXiv  Detail & Related papers  (2021-06-17T20:00:55Z)
• Improving Metric Dimensionality Reduction with Distributed Topology [68.8204255655161]
  DIPOLE is a dimensionality-reduction post-processing step that corrects an initial embedding by minimizing a loss functional with both a local, metric term and a global, topological term. We observe that DIPOLE outperforms popular methods like UMAP, t-SNE, and Isomap on a number of popular datasets.
  arXiv  Detail & Related papers  (2021-06-14T17:19:44Z)
• GELATO: Geometrically Enriched Latent Model for Offline Reinforcement Learning [54.291331971813364]
  offline reinforcement learning approaches can be divided into proximal and uncertainty-aware methods. In this work, we demonstrate the benefit of combining the two in a latent variational model. Our proposed metrics measure both the quality of out of distribution samples as well as the discrepancy of examples in the data.
  arXiv  Detail & Related papers  (2021-02-22T19:42:40Z)
• Graph Based Gaussian Processes on Restricted Domains [13.416168979487118]
  In nonparametric regression, it is common for the inputs to fall in a restricted subset of Euclidean space. We propose a new class of Graph Laplacian based GPs (GL-GPs) which learn a covariance that respects the geometry of the input domain. We provide substantial theoretical support for the GL-GP methodology, and illustrate performance gains in various applications.
  arXiv  Detail & Related papers  (2020-10-14T17:01:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.