Decoupling Pixel Flipping and Occlusion Strategy for Consistent XAI Benchmarks

• URL: http://arxiv.org/abs/2401.06654v1
• Date: Fri, 12 Jan 2024 16:01:17 GMT
• Title: Decoupling Pixel Flipping and Occlusion Strategy for Consistent XAI Benchmarks
• Abstract summary: Occlusion strategies can vary significantly from simple mean replacement up to inpainting with state-of-the-art diffusion models. PF benchmarks lead to contradicting rankings. This is amplified by competing PF measures: Features are either removed starting with most influential first (MIF) or least influential first (LIF)
• Score: 1.068128849363198
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Feature removal is a central building block for eXplainable AI (XAI), both for occlusion-based explanations (Shapley values) as well as their evaluation (pixel flipping, PF). However, occlusion strategies can vary significantly from simple mean replacement up to inpainting with state-of-the-art diffusion models. This ambiguity limits the usefulness of occlusion-based approaches. For example, PF benchmarks lead to contradicting rankings. This is amplified by competing PF measures: Features are either removed starting with most influential first (MIF) or least influential first (LIF). This study proposes two complementary perspectives to resolve this disagreement problem. Firstly, we address the common criticism of occlusion-based XAI, that artificial samples lead to unreliable model evaluations. We propose to measure the reliability by the R(eference)-Out-of-Model-Scope (OMS) score. The R-OMS score enables a systematic comparison of occlusion strategies and resolves the disagreement problem by grouping consistent PF rankings. Secondly, we show that the insightfulness of MIF and LIF is conversely dependent on the R-OMS score. To leverage this, we combine the MIF and LIF measures into the symmetric relevance gain (SRG) measure. This breaks the inherent connection to the underlying occlusion strategy and leads to consistent rankings. This resolves the disagreement problem, which we verify for a set of 40 different occlusion strategies.

Related papers

• Explaining RhythmFormer: A Systematic XAI Analysis of Periodic Sparse Attention for Remote Photoplethysmography [1.647210198730602]
  Photoplethys transformers achieve low heart-rate error on benchmarks, yet their decisions remain opaque.<n>Existing XAI is dominated by qualitative heatmap inspection without quantitative faithfulness metrics or physiology-grounded validation.<n>We adapt four methods (raw attention, rollout, flow, Beyond Intuition) to quantifying RhythmFormer's bi-level attention with top-$k$ selection.
  arXiv  Detail & Related papers  (2026-06-11T19:10:06Z)
• Learning domain-invariant features through channel-level sparsification for Out-Of Distribution Generalization [19.292845489957077]
  Out-of-Distribution generalization has become a primary metric for evaluating image analysis systems.<n>Deep learning models tend to capture domain-specific context, leading to shortcut dependencies on non-causal features.<n>We propose Hierarchical Causal Dropout (HCD), a method that uses channel-level causal masks to enforce feature sparsity.
  arXiv  Detail & Related papers  (2026-03-26T06:36:42Z)
• When Fusion Helps and When It Breaks: View-Aligned Robustness in Same-Source Financial Imaging [4.857561672498819]
  We study same-source multi-view learning and adversarial robustness for next-day direction prediction with financial image representations.<n>We construct two window-aligned views from each rolling window: an OHLCV-rendered price/volume chart and a technical-indicator matrix.<n>We evaluate test-time L-infinity perturbations using FGSM and PGD under two threat scenarios: view-constrained attacks that perturb one view and joint attacks that perturb both.
  arXiv  Detail & Related papers  (2026-02-11T16:45:23Z)
• GRCF: Two-Stage Groupwise Ranking and Calibration Framework for Multimodal Sentiment Analysis [20.77940776708036]
  Pairwise ordinal learning frameworks capture relative order by learning from comparisons.<n>They assign uniform importance to all comparisons, failing to adaptively focus on hard-to-rank samples.<n>We propose a Two-Stage Group-wise Ranking and Framework (GRCF) that adapts the philosophy of Group Relative Policy Optimization.<n>GRCF achieves state-of-the-art performance on core regression benchmarks, while also showing strong generalizability in classification tasks.
  arXiv  Detail & Related papers  (2026-01-14T16:26:44Z)
• MS-ISSM: Objective Quality Assessment of Point Clouds Using Multi-scale Implicit Structural Similarity [65.85858856481131]
  unstructured and irregular nature of point clouds poses a significant challenge for objective quality assessment (PCQA)<n>We propose the Multi-scale Implicit Structural Similarity Measurement (MS-ISSM)
  arXiv  Detail & Related papers  (2026-01-03T14:58:52Z)
• SoliReward: Mitigating Susceptibility to Reward Hacking and Annotation Noise in Video Generation Reward Models [53.19726629537694]
  Post-training alignment of video generation models with human preferences is a critical goal.<n>Current data collection paradigms, reliant on in-prompt pairwise annotations, suffer from labeling noise.<n>We propose SoliReward, a systematic framework for video RM training.
  arXiv  Detail & Related papers  (2025-12-17T14:28:23Z)
• NDCG-Consistent Softmax Approximation with Accelerated Convergence [67.10365329542365]
  We propose novel loss formulations that align directly with ranking metrics.<n>We integrate the proposed RG losses with the highly efficient Alternating Least Squares (ALS) optimization method.<n> Empirical evaluations on real-world datasets demonstrate that our approach achieves comparable or superior ranking performance.
  arXiv  Detail & Related papers  (2025-06-11T06:59:17Z)
• iEBAKER: Improved Remote Sensing Image-Text Retrieval Framework via Eliminate Before Align and Keyword Explicit Reasoning [80.44805667907612]
  iEBAKER is an innovative strategy to filter weakly correlated sample pairs. We introduce an alternative Sort After Reversed Retrieval (SAR) strategy. We incorporate a Keyword Explicit Reasoning (KER) module to facilitate the beneficial impact of subtle key concept distinctions.
  arXiv  Detail & Related papers  (2025-04-08T03:40:19Z)
• A Simple and Generalist Approach for Panoptic Segmentation [57.94892855772925]
  Generalist vision models aim for one and the same architecture for a variety of vision tasks. While such shared architecture may seem attractive, generalist models tend to be outperformed by their bespoken counterparts. We address this problem by introducing two key contributions, without compromising the desirable properties of generalist models.
  arXiv  Detail & Related papers  (2024-08-29T13:02:12Z)
• Confronting Reward Model Overoptimization with Constrained RLHF [114.71591361764547]
  We show that correlation between component RMs has a significant effect on the locations of these points. Our method addresses the problem of weighting component RMs by learning dynamic weights, naturally expressed by Lagrange multipliers.
  arXiv  Detail & Related papers  (2023-10-06T16:59:17Z)
• AFPN: Asymptotic Feature Pyramid Network for Object Detection [16.86715579071991]
  This paper proposes an feature pyramid network (AFPN) to support direct interaction at non-adjacent levels. AFPN is initiated by fusing two adjacent low-level features and achieves higher-level features into the fusion process. We incorporate the proposed AFPN into both two-stage and one-stage object detection frameworks and evaluate with the MS-COCO 2017 validation and test datasets.
  arXiv  Detail & Related papers  (2023-06-28T07:58:49Z)
• Deep Metric Learning with Soft Orthogonal Proxies [1.823505080809275]
  We propose a novel approach that introduces Soft Orthogonality (SO) constraint on proxies. Our approach leverages Data-Efficient Image Transformer (DeiT) as an encoder to extract contextual features from images along with a DML objective. Our evaluations demonstrate the superiority of our proposed approach over state-of-the-art methods by a significant margin.
  arXiv  Detail & Related papers  (2023-06-22T17:22:15Z)
• Boosting Few-shot Fine-grained Recognition with Background Suppression and Foreground Alignment [53.401889855278704]
  Few-shot fine-grained recognition (FS-FGR) aims to recognize novel fine-grained categories with the help of limited available samples. We propose a two-stage background suppression and foreground alignment framework, which is composed of a background activation suppression (BAS) module, a foreground object alignment (FOA) module, and a local to local (L2L) similarity metric. Experiments conducted on multiple popular fine-grained benchmarks demonstrate that our method outperforms the existing state-of-the-art by a large margin.
  arXiv  Detail & Related papers  (2022-10-04T07:54:40Z)
• Quality-aware Part Models for Occluded Person Re-identification [77.24920810798505]
  Occlusion poses a major challenge for person re-identification (ReID) Existing approaches typically rely on outside tools to infer visible body parts, which may be suboptimal in terms of both computational efficiency and ReID accuracy. We propose a novel method named Quality-aware Part Models (QPM) for occlusion-robust ReID.
  arXiv  Detail & Related papers  (2022-01-01T03:51:09Z)
• Learning to Estimate Hidden Motions with Global Motion Aggregation [71.12650817490318]
  Occlusions pose a significant challenge to optical flow algorithms that rely on local evidences. We introduce a global motion aggregation module to find long-range dependencies between pixels in the first image. We demonstrate that the optical flow estimates in the occluded regions can be significantly improved without damaging the performance in non-occluded regions.
  arXiv  Detail & Related papers  (2021-04-06T10:32:03Z)
• Policy Evaluation and Seeking for Multi-Agent Reinforcement Learning via Best Response [15.149039407681945]
  We adopt strict best response dynamics to model selfish behaviors at a meta-level for multi-agent reinforcement learning. Our approach is more compatible with single-agent reinforcement learning than alpha-rank which relies on weakly better responses.
  arXiv  Detail & Related papers  (2020-06-17T01:17:52Z)
• Enhancing Geometric Factors in Model Learning and Inference for Object Detection and Instance Segmentation [91.12575065731883]
  We propose Complete-IoU (CIoU) loss and Cluster-NMS for enhancing geometric factors in both bounding box regression and Non-Maximum Suppression (NMS) The training of deep models using CIoU loss results in consistent AP and AR improvements in comparison to widely adopted $ell_n$-norm loss and IoU-based loss. Cluster-NMS is very efficient due to its pure GPU implementation, and geometric factors can be incorporated to improve both AP and AR.
  arXiv  Detail & Related papers  (2020-05-07T16:00:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.