Identifying Important Group of Pixels using Interactions

• URL: http://arxiv.org/abs/2401.03785v3
• Date: Thu, 15 Aug 2024 12:17:11 GMT
• Title: Identifying Important Group of Pixels using Interactions
• Authors: Kosuke Sumiyasu, Kazuhiko Kawamoto, Hiroshi Kera,
• Abstract summary: We propose a method, MoXI, that efficiently identifies a group of pixels with high prediction confidence. The proposed method employs game-theoretic concepts, Shapley values and interactions, taking into account the effects of individual pixels.
• Score: 5.2980803808373516
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: To better understand the behavior of image classifiers, it is useful to visualize the contribution of individual pixels to the model prediction. In this study, we propose a method, MoXI ($\textbf{Mo}$del e$\textbf{X}$planation by $\textbf{I}$nteractions), that efficiently and accurately identifies a group of pixels with high prediction confidence. The proposed method employs game-theoretic concepts, Shapley values and interactions, taking into account the effects of individual pixels and the cooperative influence of pixels on model confidence. Theoretical analysis and experiments demonstrate that our method better identifies the pixels that are highly contributing to the model outputs than widely-used visualization by Grad-CAM, Attention rollout, and Shapley value. While prior studies have suffered from the exponential computational cost in the computation of Shapley value and interactions, we show that this can be reduced to quadratic cost for our task. The code is available at https://github.com/KosukeSumiyasu/MoXI.

Related papers

• Bayesian Active Learning for Semantic Segmentation [9.617769135242973]
  We introduce a Bayesian active learning framework based on sparse pixel-level annotation. BalEnt captures the information between the models' predicted marginalized probability distribution and the pixel labels. We train our proposed active learning framework for Cityscapes, Camvid, ADE20K and VOC2012 benchmark datasets.
  arXiv  Detail & Related papers  (2024-08-03T07:26:10Z)
• Semi-supervised Counting via Pixel-by-pixel Density Distribution Modelling [135.66138766927716]
  This paper focuses on semi-supervised crowd counting, where only a small portion of the training data are labeled. We formulate the pixel-wise density value to regress as a probability distribution, instead of a single deterministic value. Our method clearly outperforms the competitors by a large margin under various labeled ratio settings.
  arXiv  Detail & Related papers  (2024-02-23T12:48:02Z)
• Learning county from pixels: Corn yield prediction with attention-weighted multiple instance learning [8.573309028586168]
  This research examines each county at the pixel level and applies multiple instance learning to leverage detailed information within a county. In addition, our method addresses the "mixed pixel" issue caused by the inconsistent resolution between feature datasets and crop mask. The developed model outperforms four other machine learning models over the past five years in the U.S. corn belt.
  arXiv  Detail & Related papers  (2023-12-02T02:09:31Z)
• Pixel-Inconsistency Modeling for Image Manipulation Localization [59.968362815126326]
  Digital image forensics plays a crucial role in image authentication and manipulation localization. This paper presents a generalized and robust manipulation localization model through the analysis of pixel inconsistency artifacts. Experiments show that our method successfully extracts inherent pixel-inconsistency forgery fingerprints.
  arXiv  Detail & Related papers  (2023-09-30T02:54:51Z)
• Superpixels algorithms through network community detection [0.0]
  Community detection is a powerful tool from complex networks analysis that finds applications in various research areas. Superpixels aim at representing the image at a smaller level while preserving as much as possible original information. We study the efficiency of superpixels computed by state-of-the-art community detection algorithms on a 4-connected pixel graph.
  arXiv  Detail & Related papers  (2023-08-27T13:13:28Z)
• Autoencoding Conditional Neural Processes for Representation Learning [31.63717849083666]
  Conditional neural processes (CNPs) are a flexible and efficient family of models that learn to learn from data. We develop the Partial Pixel Space Variational Autoencoder (PPS-VAE), an amortised variational framework that casts CNP context as latent variables learnt simultaneously with the CNP.
  arXiv  Detail & Related papers  (2023-05-29T12:10:18Z)
• Single Image Depth Prediction Made Better: A Multivariate Gaussian Take [163.14849753700682]
  We introduce an approach that performs continuous modeling of per-pixel depth. Our method's accuracy (named MG) is among the top on the KITTI depth-prediction benchmark leaderboard.
  arXiv  Detail & Related papers  (2023-03-31T16:01:03Z)
• Probabilistic Deep Metric Learning for Hyperspectral Image Classification [91.5747859691553]
  This paper proposes a probabilistic deep metric learning framework for hyperspectral image classification. It aims to predict the category of each pixel for an image captured by hyperspectral sensors. Our framework can be readily applied to existing hyperspectral image classification methods.
  arXiv  Detail & Related papers  (2022-11-15T17:57:12Z)
• Pixel-Wise Contrastive Distillation [3.274323556083613]
  We present a pixel-level self-supervised distillation framework friendly to dense prediction tasks. Our method, called Pixel-Wise Contrastive Distillation (PCD), distills knowledge by attracting the corresponding pixels from student's and teacher's output feature maps.
  arXiv  Detail & Related papers  (2022-11-01T02:00:32Z)
• PixelPyramids: Exact Inference Models from Lossless Image Pyramids [58.949070311990916]
  Pixel-Pyramids is a block-autoregressive approach with scale-specific representations to encode the joint distribution of image pixels. It yields state-of-the-art results for density estimation on various image datasets, especially for high-resolution data. For CelebA-HQ 1024 x 1024, we observe that the density estimates are improved to 44% of the baseline despite sampling speeds superior even to easily parallelizable flow-based models.
  arXiv  Detail & Related papers  (2021-10-17T10:47:29Z)
• ITSELF: Iterative Saliency Estimation fLexible Framework [68.8204255655161]
  Saliency object detection estimates the objects that most stand out in an image. We propose a superpixel-based ITerative Saliency Estimation fLexible Framework (ITSELF) that allows any user-defined assumptions to be added to the model. We compare ITSELF to two state-of-the-art saliency estimators on five metrics and six datasets.
  arXiv  Detail & Related papers  (2020-06-30T16:51:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.