AGOP as Explanation: From Feature Learning to Per-Sample Attribution in Image Classifiers

• URL: http://arxiv.org/abs/2605.12816v1
• Date: Tue, 12 May 2026 23:15:47 GMT
• Title: AGOP as Explanation: From Feature Learning to Per-Sample Attribution in Image Classifiers
• Abstract summary: We introduce AGOP-Weighted: a novel attribution method that multiplies the per-sample gradient by sqrt(diag(M) / max diag(M)<n>We conduct the first rigorous comparison of AGOP attribution against Integrated Gradients (IG), SmoothGrad, GradCAM, and VanillaGrad.<n>AGOP-Weighted achieves 44% higher mIoU than IG on linear tasks; AGOP-Global achieves 7x higher mIoU than IG on multiplicative tasks.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The Average Gradient Outer Product (AGOP) governs feature learning in neural networks: the Neural Feature Ansatz states that weight Gram matrices at each layer align with the corresponding AGOP matrices computed over the training distribution. We ask a complementary question: can this same quantity serve as a post-hoc attribution method for explaining individual predictions? We introduce AGOP-Weighted: a novel attribution method that multiplies the per-sample gradient by sqrt(diag(M) / max diag(M)), a training-distribution prior that suppresses gradient noise and amplifies consistently important pixels -- a combination not present in any prior attribution method. We formalise two companion variants -- AGOP-Local (per-sample gradient, equivalent to VanillaGrad) and AGOP-Global (diag(M) directly as a zero-cost saliency map) -- and implement an efficient training-time accumulation hook; AGOP-Global then requires zero inference cost (disk lookup) while AGOP-Weighted requires only a single gradient pass. We conduct the first rigorous comparison of AGOP attribution against Integrated Gradients (IG), SmoothGrad, GradCAM, and VanillaGrad across two benchmarks with pixel-level ground truth: (i) the synthetic XAI-TRIS benchmark (four classification scenarios, 8x8 images, CNN8by8) and (ii) the photorealistic CLEVR-XAI benchmark (ResNet-18 fine-tuned from ImageNet). AGOP-Weighted achieves 44% higher mIoU than IG on linear tasks; AGOP-Global achieves 7x higher mIoU than IG on multiplicative tasks (where IG falls below random) at zero inference cost. Both findings generalise to ResNet-18 on CLEVR-XAI (+18% and +37% respectively). We further show that GradCAM fails on small-resolution images due to spatial resolution collapse, and that diag(M) quality improves monotonically throughout training even after classification accuracy has plateaued.

Related papers

• Global Convergence of Gradient EM for Over-Parameterized Gaussian Mixtures [53.51230405648361]
  We study the dynamics of gradient EM and employ tensor decomposition to characterize the geometric landscape of the likelihood loss.<n>This is the first global convergence and recovery result for EM or gradient EM beyond the special case of $m=2$.
  arXiv  Detail & Related papers  (2025-06-06T23:32:38Z)
• Generalized and Efficient 2D Gaussian Splatting for Arbitrary-scale Super-Resolution [10.074968164380314]
  Implicit Neural Representations (INR) have been successfully employed for Arbitrary-scale Super-Resolution (ASR)<n>However, INR-based models need to query the multi-layer perceptron module numerous times and render a pixel in each query.<n>GS has shown its advantages over INR in both visual quality and rendering speed in 3D tasks, which motivates us to explore whether GS can be employed for the ASR task.
  arXiv  Detail & Related papers  (2025-01-12T15:14:58Z)
• DELT: A Simple Diversity-driven EarlyLate Training for Dataset Distillation [23.02066055996762]
  We propose a new Diversity-driven EarlyLate Training (DELT) scheme to enhance the diversity of images in batch-to-global matching.<n>Our approach is conceptually simple yet effective, it partitions predefined IPC samples into smaller subtasks and employs local optimizations.<n>Our approach outperforms the previous state-of-the-art by 2$sim$5% on average across different datasets and IPCs (images per class)
  arXiv  Detail & Related papers  (2024-11-29T18:59:46Z)
• Gradient-Free Classifier Guidance for Diffusion Model Sampling [4.450496470631169]
  Gradient-free Guidance (GFCG) method consistently improves class prediction accuracy. For ImageNet 512$times$512, we achieve a record $FD_textDINOv2$ 23.09, while simultaneously attaining a higher classification Precision (94.3%) compared to ATG (90.2%)
  arXiv  Detail & Related papers  (2024-11-23T00:22:21Z)
• MGDA Converges under Generalized Smoothness, Provably [27.87166415148172]
  Multi-objective optimization (MOO) is receiving more attention in various fields such as multi-task learning.<n>Recent works provide some effective algorithms with theoretical analysis but they are limited by the standard $L$-smooth or bounded-gradient assumptions.<n>We study a more general and realistic class of generalized $ell$-smooth loss functions, where $ell$ is a general non-decreasing function of gradient norm.
  arXiv  Detail & Related papers  (2024-05-29T18:36:59Z)
• Cramer Type Distances for Learning Gaussian Mixture Models by Gradient Descent [0.0]
  As of today, few known algorithms can fit or learn Gaussian mixture models. We propose a distance function called Sliced Cram'er 2-distance for learning general multivariate GMMs. These features are especially useful for distributional reinforcement learning and Deep Q Networks.
  arXiv  Detail & Related papers  (2023-07-13T13:43:02Z)
• High-fidelity Pseudo-labels for Boosting Weakly-Supervised Segmentation [17.804090651425955]
  Image-level weakly-supervised segmentation (WSSS) reduces the usually vast data annotation cost by surrogate segmentation masks during training. Our work is based on two techniques for improving CAMs; importance sampling, which is a substitute for GAP, and the feature similarity loss. We reformulate both techniques based on binomial posteriors of multiple independent binary problems. This has two benefits; their performance is improved and they become more general, resulting in an add-on method that can boost virtually any WSSS method.
  arXiv  Detail & Related papers  (2023-04-05T17:43:57Z)
• Cap2Aug: Caption guided Image to Image data Augmentation [41.53127698828463]
  Cap2Aug is an image-to-image diffusion model-based data augmentation strategy using image captions as text prompts. We generate captions from the limited training images and using these captions edit the training images using an image-to-image stable diffusion model. This strategy generates augmented versions of images similar to the training images yet provides semantic diversity across the samples.
  arXiv  Detail & Related papers  (2022-12-11T04:37:43Z)
• Noise Self-Regression: A New Learning Paradigm to Enhance Low-Light Images Without Task-Related Data [86.68013790656762]
  We propose Noise SElf-Regression (NoiSER) without access to any task-related data.<n>NoiSER is highly competitive in enhancement quality, yet with a much smaller model size, and much lower training and inference cost.
  arXiv  Detail & Related papers  (2022-11-09T06:18:18Z)
• Scale Attention for Learning Deep Face Representation: A Study Against Visual Scale Variation [69.45176408639483]
  We reform the conv layer by resorting to the scale-space theory. We build a novel style named SCale AttentioN Conv Neural Network (textbfSCAN-CNN) As a single-shot scheme, the inference is more efficient than multi-shot fusion.
  arXiv  Detail & Related papers  (2022-09-19T06:35:04Z)
• Semi-Supervised Domain Adaptation with Prototypical Alignment and Consistency Learning [86.6929930921905]
  This paper studies how much it can help address domain shifts if we further have a few target samples labeled. To explore the full potential of landmarks, we incorporate a prototypical alignment (PA) module which calculates a target prototype for each class from the landmarks. Specifically, we severely perturb the labeled images, making PA non-trivial to achieve and thus promoting model generalizability.
  arXiv  Detail & Related papers  (2021-04-19T08:46:08Z)
• Permuted AdaIN: Reducing the Bias Towards Global Statistics in Image Classification [97.81205777897043]
  Recent work has shown that convolutional neural network classifiers overly rely on texture at the expense of shape cues. We make a similar but different distinction between shape and local image cues, on the one hand, and global image statistics, on the other. Our method, called Permuted Adaptive Instance Normalization (pAdaIN), reduces the representation of global statistics in the hidden layers of image classifiers.
  arXiv  Detail & Related papers  (2020-10-09T16:38:38Z)
• Towards Better Understanding of Adaptive Gradient Algorithms in Generative Adversarial Nets [71.05306664267832]
  Adaptive algorithms perform gradient updates using the history of gradients and are ubiquitous in training deep neural networks. In this paper we analyze a variant of OptimisticOA algorithm for nonconcave minmax problems. Our experiments show that adaptive GAN non-adaptive gradient algorithms can be observed empirically.
  arXiv  Detail & Related papers  (2019-12-26T22:10:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.