On Spectral Properties of Gradient-based Explanation Methods

• URL: http://arxiv.org/abs/2508.10595v1
• Date: Thu, 14 Aug 2025 12:37:22 GMT
• Title: On Spectral Properties of Gradient-based Explanation Methods
• Authors: Amir Mehrpanah, Erik Englesson, Hossein Azizpour,
• Abstract summary: We adopt novel probabilistic and spectral perspectives to analyze explanation methods.<n>Our study reveals a pervasive spectral bias stemming from the use of gradient, and sheds light on some common design choices.<n>We propose two remedies based on our proposed formalism: (i) a mechanism to determine a standard perturbation scale, and (ii) an aggregation method which we call SpectralLens.
• Score: 6.181300669254824
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Understanding the behavior of deep networks is crucial to increase our confidence in their results. Despite an extensive body of work for explaining their predictions, researchers have faced reliability issues, which can be attributed to insufficient formalism. In our research, we adopt novel probabilistic and spectral perspectives to formally analyze explanation methods. Our study reveals a pervasive spectral bias stemming from the use of gradient, and sheds light on some common design choices that have been discovered experimentally, in particular, the use of squared gradient and input perturbation. We further characterize how the choice of perturbation hyperparameters in explanation methods, such as SmoothGrad, can lead to inconsistent explanations and introduce two remedies based on our proposed formalism: (i) a mechanism to determine a standard perturbation scale, and (ii) an aggregation method which we call SpectralLens. Finally, we substantiate our theoretical results through quantitative evaluations.

Related papers

• Revisiting Zeroth-Order Optimization: Minimum-Variance Two-Point Estimators and Directionally Aligned Perturbations [57.179679246370114]
  We identify the distribution of random perturbations that minimizes the estimator's variance as the perturbation stepsize tends to zero.<n>Our findings reveal that such desired perturbations can align directionally with the true gradient, instead of maintaining a fixed length.
  arXiv  Detail & Related papers  (2025-10-22T19:06:39Z)
• On the Complexity-Faithfulness Trade-off of Gradient-Based Explanations [5.528734654854472]
  ReLU networks have sharp transitions, sometimes relying on individual pixels for predictions.<n>Existing methods, such as GradCAM, smooth these explanations by producing surrogate models at the cost of faithfulness.<n>We introduce a unifying spectral framework to systematically analyze and quantify smoothness, faithfulness, and their trade-off in explanations.
  arXiv  Detail & Related papers  (2025-08-14T09:49:07Z)
• Avoided-crossings, degeneracies and Berry phases in the spectrum of quantum noise through analytic Bloch-Messiah decomposition [49.1574468325115]
  "analytic Bloch-Messiah decomposition" provides approach for characterizing dynamics of quantum optical systems.<n>We show that avoided crossings arise naturally when a single parameter is varied, leading to hypersensitivity of the singular vectors.<n>We highlight the possibility of programming the spectral response of photonic systems through the deliberate design of avoided crossings.
  arXiv  Detail & Related papers  (2025-04-29T13:14:15Z)
• Towards Understanding the Optimization Mechanisms in Deep Learning [5.281849820329249]
  In this paper, we adopt a distribution estimation perspective to explore the mechanisms of supervised classification using deep neural networks.<n>For the latter, we provide theoretical insights into mechanisms such as over- and probability randomization.
  arXiv  Detail & Related papers  (2025-03-29T08:46:13Z)
• Spectral Analysis of Diffusion Models with Application to Schedule Design [23.105365495914644]
  Diffusion models (DMs) have emerged as powerful tools for modeling complex data distributions.<n>We offer a novel analysis of the DM's inference process, introducing a comprehensive frequency response perspective.<n>We demonstrate how the proposed analysis can be leveraged to design a noise schedule that aligns effectively with the characteristics of the data.
  arXiv  Detail & Related papers  (2025-01-31T21:50:31Z)
• Explaining Predictive Uncertainty by Exposing Second-Order Effects [13.83164409095901]
  We present a new method for explaining predictive uncertainty based on second-order effects. Our method is generally applicable, allowing for turning common attribution techniques into powerful second-order uncertainty explainers.
  arXiv  Detail & Related papers  (2024-01-30T21:02:21Z)
• Spectral Decomposition Representation for Reinforcement Learning [100.0424588013549]
  We propose an alternative spectral method, Spectral Decomposition Representation (SPEDER), that extracts a state-action abstraction from the dynamics without inducing spurious dependence on the data collection policy. A theoretical analysis establishes the sample efficiency of the proposed algorithm in both the online and offline settings. An experimental investigation demonstrates superior performance over current state-of-the-art algorithms across several benchmarks.
  arXiv  Detail & Related papers  (2022-08-19T19:01:30Z)
• On the Benefits of Large Learning Rates for Kernel Methods [110.03020563291788]
  We show that a phenomenon can be precisely characterized in the context of kernel methods. We consider the minimization of a quadratic objective in a separable Hilbert space, and show that with early stopping, the choice of learning rate influences the spectral decomposition of the obtained solution.
  arXiv  Detail & Related papers  (2022-02-28T13:01:04Z)
• Neural density estimation and uncertainty quantification for laser induced breakdown spectroscopy spectra [4.698576003197588]
  We use normalizing flows on structured spectral latent spaces to estimate probability densities. We evaluate a method for uncertainty quantification when predicting unobserved state vectors. We demonstrate the capability of this approach on laser-induced breakdown spectroscopy data collected by the Mars rover Curiosity.
  arXiv  Detail & Related papers  (2021-08-17T01:10:29Z)
• Discovering Latent Causal Variables via Mechanism Sparsity: A New Principle for Nonlinear ICA [81.4991350761909]
  Independent component analysis (ICA) refers to an ensemble of methods which formalize this goal and provide estimation procedure for practical application. We show that the latent variables can be recovered up to a permutation if one regularizes the latent mechanisms to be sparse.
  arXiv  Detail & Related papers  (2021-07-21T14:22:14Z)
• Leveraging Global Parameters for Flow-based Neural Posterior Estimation [90.21090932619695]
  Inferring the parameters of a model based on experimental observations is central to the scientific method. A particularly challenging setting is when the model is strongly indeterminate, i.e., when distinct sets of parameters yield identical observations. We present a method for cracking such indeterminacy by exploiting additional information conveyed by an auxiliary set of observations sharing global parameters.
  arXiv  Detail & Related papers  (2021-02-12T12:23:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.