Decision-Aware Attention Propagation for Vision Transformer Explainability

• URL: http://arxiv.org/abs/2604.18094v1
• Date: Mon, 20 Apr 2026 11:10:18 GMT
• Title: Decision-Aware Attention Propagation for Vision Transformer Explainability
• Abstract summary: Decision-Aware Attention Propagation (DAP) is an attribution method that injects decision-relevant priors into transformer attention propagation.<n>DAP produces attribution maps that are more class sensitive, compact, and faithful than those generated by conventional attention based methods.
• Score: 1.9116784879310027
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision Transformers (ViTs) have become a dominant architecture in computer vision, yet their prediction process remains difficult to interpret because information is propagated through complex interactions across layers and attention heads. Existing attention based explanation methods provide an intuitive way to trace information flow. However, they rely mainly on raw attention weights, which do not explicitly reflect the final decision and often lead to explanations with limited class discriminability. In contrast, gradient based localization methods are more effective at highlighting class specific evidence, but they do not fully exploit the hierarchical attention propagation mechanism of transformers. To address this limitation, we propose Decision-Aware Attention Propagation (DAP), an attribution method that injects decision-relevant priors into transformer attention propagation. By estimating token importance through gradient based localization and integrating it into layer wise attention rollout, the method captures both the structural flow of attention and the evidence most relevant to the final prediction. Consequently, DAP produces attribution maps that are more class sensitive, compact, and faithful than those generated by conventional attention based methods. Extensive experiments across Vision Transformer variants of different model scales show that DAP consistently outperforms existing baselines in both quantitative metrics and qualitative visualizations, indicating that decision aware propagation is an effective direction for improving ViT interpretability.

Related papers

• Gradient-Skipping Relevance Propagation for Efficient Explainability of Vision Transformers [5.770347328961063]
  We introduce GradSkip, a novel relevance propagation method for ViTs based on adaptive head weighting and skip-aware propagation.<n>Experiments on ImageNet1K and BloodMNIST demonstrate GradSkip's faithfulness while requiring over 14 times fewer GFLOPs than the best-performing existing approaches.
  arXiv  Detail & Related papers  (2026-07-11T15:43:09Z)
• Learn to Rank: Visual Attribution by Learning Importance Ranking [58.69028273772474]
  We propose a new approach to generate visual attribution maps for computer vision models.<n>Our method produces dense, pixel-level attributions in a single forward pass with optional, few-step gradient refinement.<n>Our experiments demonstrate consistent quantitative improvements and sharper, boundary-aligned explanations.
  arXiv  Detail & Related papers  (2026-04-07T12:53:22Z)
• Where and How to Perturb: On the Design of Perturbation Guidance in Diffusion and Flow Models [48.385105851598745]
  "HeadHunter" is a systematic framework for iteratively selecting attention heads that align with user-centric objectives.<n>"SoftPAG" is a continuous knob to tune perturbation strength and suppress artifacts.<n>We validate our method on modern large-scale DiT-based text-to-image models.
  arXiv  Detail & Related papers  (2025-06-12T17:59:51Z)
• DAPE V2: Process Attention Score as Feature Map for Length Extrapolation [63.87956583202729]
  We conceptualize attention as a feature map and apply the convolution operator to mimic the processing methods in computer vision. The novel insight, which can be adapted to various attention-related models, reveals that the current Transformer architecture has the potential for further evolution.
  arXiv  Detail & Related papers  (2024-10-07T07:21:49Z)
• Noise-Free Explanation for Driving Action Prediction [11.330363757618379]
  We propose an easy-to-implement but effective way to remedy this flaw: Smooth Noise Norm Attention (SNNA) We weigh the attention by the norm of the transformed value vector and guide the label-specific signal with the attention gradient, then randomly sample the input perturbations and average the corresponding gradients to produce noise-free attribution. Both qualitative and quantitative evaluation results show the superiority of SNNA compared to other SOTA attention-based explainable methods in generating a clearer visual explanation map and ranking the input pixel importance.
  arXiv  Detail & Related papers  (2024-07-08T19:21:24Z)
• Visualizing Transferred Knowledge: An Interpretive Model of Unsupervised Domain Adaptation [70.85686267987744]
  Unsupervised domain adaptation problems can transfer knowledge from a labeled source domain to an unlabeled target domain. We propose an interpretive model of unsupervised domain adaptation, as the first attempt to visually unveil the mystery of transferred knowledge. Our method provides an intuitive explanation for the base model's predictions and unveils transfer knowledge by matching the image patches with the same semantics across both source and target domains.
  arXiv  Detail & Related papers  (2023-03-04T03:02:12Z)
• XAI for Transformers: Better Explanations through Conservative Propagation [60.67748036747221]
  We show that the gradient in a Transformer reflects the function only locally, and thus fails to reliably identify the contribution of input features to the prediction. Our proposal can be seen as a proper extension of the well-established LRP method to Transformers.
  arXiv  Detail & Related papers  (2022-02-15T10:47:11Z)
• Transformer Interpretability Beyond Attention Visualization [87.96102461221415]
  Self-attention techniques, and specifically Transformers, are dominating the field of text processing. In this work, we propose a novel way to compute relevancy for Transformer networks.
  arXiv  Detail & Related papers  (2020-12-17T18:56:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.