BEE: Metric-Adapted Explanations via Baseline Exploration-Exploitation

• URL: http://arxiv.org/abs/2412.17512v1
• Date: Mon, 23 Dec 2024 12:19:03 GMT
• Title: BEE: Metric-Adapted Explanations via Baseline Exploration-Exploitation
• Authors: Oren Barkan, Yehonatan Elisha, Jonathan Weill, Noam Koenigstein,
• Abstract summary: Two prominent challenges in explainability research involve 1) the nuanced evaluation of explanations and 2) the modeling of missing information. We propose Baseline Exploration-Exploitation (BEE) - a path-integration method that introduces randomness to the integration process. BEE generates a comprehensive set of explanation maps, facilitating the selection of the best-performing explanation map.
• Score: 10.15605247436119
• License:
• Abstract: Two prominent challenges in explainability research involve 1) the nuanced evaluation of explanations and 2) the modeling of missing information through baseline representations. The existing literature introduces diverse evaluation metrics, each scrutinizing the quality of explanations through distinct lenses. Additionally, various baseline representations have been proposed, each modeling the notion of missingness differently. Yet, a consensus on the ultimate evaluation metric and baseline representation remains elusive. This work acknowledges the diversity in explanation metrics and baselines, demonstrating that different metrics exhibit preferences for distinct explanation maps resulting from the utilization of different baseline representations and distributions. To address the diversity in metrics and accommodate the variety of baseline representations in a unified manner, we propose Baseline Exploration-Exploitation (BEE) - a path-integration method that introduces randomness to the integration process by modeling the baseline as a learned random tensor. This tensor follows a learned mixture of baseline distributions optimized through a contextual exploration-exploitation procedure to enhance performance on the specific metric of interest. By resampling the baseline from the learned distribution, BEE generates a comprehensive set of explanation maps, facilitating the selection of the best-performing explanation map in this broad set for the given metric. Extensive evaluations across various model architectures showcase the superior performance of BEE in comparison to state-of-the-art explanation methods on a variety of objective evaluation metrics.

Related papers

• A Unifying Information-theoretic Perspective on Evaluating Generative Models [5.524685807042777]
  Several recent approaches utilize "precision" and "recall," borrowed from the classification domain, to individually quantify the output fidelity (realism) and output diversity (representation of the real data variation) We unify a class of kth-nearest-neighbors (kNN)-based metrics under an information-theoretic lens using approaches from kNN density estimation. We propose a tri-dimensional metric composed of Precision Cross-Entropy (PCE), Recall Cross-Entropy (RCE), and Recall Entropy (RE)
  arXiv  Detail & Related papers  (2024-12-18T21:17:02Z)
• Axiomatic Explainer Globalness via Optimal Transport [3.2203930850666675]
  We define a complexity measure for explainers, globalness, which enables deeper understanding of the distribution of explanations. We prove that our proposed measure, Wasserstein Globalness, meets these criteria.
  arXiv  Detail & Related papers  (2024-11-02T04:01:38Z)
• Revealing Multimodal Contrastive Representation Learning through Latent Partial Causal Models [85.67870425656368]
  We introduce a unified causal model specifically designed for multimodal data. We show that multimodal contrastive representation learning excels at identifying latent coupled variables. Experiments demonstrate the robustness of our findings, even when the assumptions are violated.
  arXiv  Detail & Related papers  (2024-02-09T07:18:06Z)
• Understanding Probe Behaviors through Variational Bounds of Mutual Information [53.520525292756005]
  We provide guidelines for linear probing by constructing a novel mathematical framework leveraging information theory. First, we connect probing with the variational bounds of mutual information (MI) to relax the probe design, equating linear probing with fine-tuning. We show that the intermediate representations can have the biggest MI estimate because of the tradeoff between better separability and decreasing MI.
  arXiv  Detail & Related papers  (2023-12-15T18:38:18Z)
• COSE: A Consistency-Sensitivity Metric for Saliency on Image Classification [21.3855970055692]
  We present a set of metrics that utilize vision priors to assess the performance of saliency methods on image classification tasks. We show that although saliency methods are thought to be architecture-independent, most methods could better explain transformer-based models over convolutional-based models.
  arXiv  Detail & Related papers  (2023-09-20T01:06:44Z)
• Enriching Disentanglement: From Logical Definitions to Quantitative Metrics [59.12308034729482]
  Disentangling the explanatory factors in complex data is a promising approach for data-efficient representation learning. We establish relationships between logical definitions and quantitative metrics to derive theoretically grounded disentanglement metrics. We empirically demonstrate the effectiveness of the proposed metrics by isolating different aspects of disentangled representations.
  arXiv  Detail & Related papers  (2023-05-19T08:22:23Z)
• Evaluating the Robustness of Interpretability Methods through Explanation Invariance and Equivariance [72.50214227616728]
  Interpretability methods are valuable only if their explanations faithfully describe the explained model. We consider neural networks whose predictions are invariant under a specific symmetry group.
  arXiv  Detail & Related papers  (2023-04-13T17:59:03Z)
• Partial Shape Similarity via Alignment of Multi-Metric Hamiltonian Spectra [10.74981839055037]
  We propose a novel axiomatic method to match similar regions across shapes. Matching similar regions is formulated as the alignment of the spectra of operators closely related to the Laplace-Beltrami operator (LBO) We show that matching these dual spectra outperforms competing axiomatic frameworks when tested on standard benchmarks.
  arXiv  Detail & Related papers  (2022-07-07T00:03:50Z)
• The Solvability of Interpretability Evaluation Metrics [7.3709604810699085]
  Feature attribution methods are often evaluated on metrics such as comprehensiveness and sufficiency. In this paper, we highlight an intriguing property of these metrics: their solvability. We present a series of investigations showing that this beam search explainer is generally comparable or favorable to current choices.
  arXiv  Detail & Related papers  (2022-05-18T02:52:03Z)
• Semantic Change Detection with Asymmetric Siamese Networks [71.28665116793138]
  Given two aerial images, semantic change detection aims to locate the land-cover variations and identify their change types with pixel-wise boundaries. This problem is vital in many earth vision related tasks, such as precise urban planning and natural resource management. We present an asymmetric siamese network (ASN) to locate and identify semantic changes through feature pairs obtained from modules of widely different structures.
  arXiv  Detail & Related papers  (2020-10-12T13:26:30Z)
• Evaluation Metrics for Conditional Image Generation [100.69766435176557]
  We present two new metrics for evaluating generative models in the class-conditional image generation setting. A theoretical analysis shows the motivation behind each proposed metric and links the novel metrics to their unconditional counterparts. We provide an extensive empirical evaluation, comparing the metrics to their unconditional variants and to other metrics, and utilize them to analyze existing generative models.
  arXiv  Detail & Related papers  (2020-04-26T12:15:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.