Decomposing Global Feature Effects Based on Feature Interactions

• URL: http://arxiv.org/abs/2306.00541v2
• Date: Mon, 1 Jul 2024 14:26:49 GMT
• Title: Decomposing Global Feature Effects Based on Feature Interactions
• Authors: Julia Herbinger, Marvin N. Wright, Thomas Nagler, Bernd Bischl, Giuseppe Casalicchio,
• Abstract summary: Generalized additive decomposition of global effects (GADGET) is a new framework for finding interpretable regions in the feature space. We provide a mathematical foundation of the framework and show that it is applicable to the most popular methods to visualize marginal feature effects. We empirically evaluate the theoretical characteristics of the proposed methods based on various feature effect methods in different experimental settings.
• Score: 10.874932625841257
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Global feature effect methods, such as partial dependence plots, provide an intelligible visualization of the expected marginal feature effect. However, such global feature effect methods can be misleading, as they do not represent local feature effects of single observations well when feature interactions are present. We formally introduce generalized additive decomposition of global effects (GADGET), which is a new framework based on recursive partitioning to find interpretable regions in the feature space such that the interaction-related heterogeneity of local feature effects is minimized. We provide a mathematical foundation of the framework and show that it is applicable to the most popular methods to visualize marginal feature effects, namely partial dependence, accumulated local effects, and Shapley additive explanations (SHAP) dependence. Furthermore, we introduce and validate a new permutation-based interaction test to detect significant feature interactions that is applicable to any feature effect method that fits into our proposed framework. We empirically evaluate the theoretical characteristics of the proposed methods based on various feature effect methods in different experimental settings. Moreover, we apply our introduced methodology to three real-world examples to showcase their usefulness.

Related papers

• Disentangling Interactions and Dependencies in Feature Attribution [9.442326245744916]
  In machine learning, global feature importance methods try to determine how much each individual feature contributes to predicting a target variable. In commonly used feature importance scores these cooperative effects are conflated with the features' individual contributions. We derive DIP, a new mathematical decomposition of individual feature importance scores that disentangles three components.
  arXiv  Detail & Related papers  (2024-10-31T09:41:10Z)
• CAFE: Conflict-Aware Feature-wise Explanations [12.428277452418621]
  Feature attribution methods are widely used to explain neural models by determining the influence of individual input features on the models' outputs. We propose a novel feature attribution method, CAFE (Conflict-Aware Feature-wise Explanations), that addresses three limitations of the existing methods.
  arXiv  Detail & Related papers  (2023-10-31T11:14:26Z)
• Relational Local Explanations [11.679389861042]
  We develop a novel model-agnostic and permutation-based feature attribution algorithm based on relational analysis between input variables. We are able to gain a broader insight into machine learning model decisions and data.
  arXiv  Detail & Related papers  (2022-12-23T14:46:23Z)
• CausalBench: A Large-scale Benchmark for Network Inference from Single-cell Perturbation Data [61.088705993848606]
  We introduce CausalBench, a benchmark suite for evaluating causal inference methods on real-world interventional data. CaulBench incorporates biologically-motivated performance metrics, including new distribution-based interventional metrics.
  arXiv  Detail & Related papers  (2022-10-31T13:04:07Z)
• REPID: Regional Effect Plots with implicit Interaction Detection [0.9023847175654603]
  Interpretable machine learning methods visualize marginal feature effects but may lead to misleading interpretations when feature interactions are present. We introduce implicit interaction detection, a novel framework to detect interactions between a feature of interest and other features. The framework also quantifies the strength of interactions and provides interpretable and distinct regions in which feature effects can be interpreted more reliably.
  arXiv  Detail & Related papers  (2022-02-15T08:54:00Z)
• Masked Transformer for Neighhourhood-aware Click-Through Rate Prediction [74.52904110197004]
  We propose Neighbor-Interaction based CTR prediction, which put this task into a Heterogeneous Information Network (HIN) setting. In order to enhance the representation of the local neighbourhood, we consider four types of topological interaction among the nodes. We conduct comprehensive experiments on two real world datasets and the experimental results show that our proposed method outperforms state-of-the-art CTR models significantly.
  arXiv  Detail & Related papers  (2022-01-25T12:44:23Z)
• Variational Disentanglement for Domain Generalization [68.85458536180437]
  We propose to tackle the problem of domain generalization by delivering an effective framework named Variational Disentanglement Network (VDN) VDN is capable of disentangling the domain-specific features and task-specific features, where the task-specific features are expected to be better generalized to unseen but related test data.
  arXiv  Detail & Related papers  (2021-09-13T09:55:32Z)
• On Inductive Biases for Heterogeneous Treatment Effect Estimation [91.3755431537592]
  We investigate how to exploit structural similarities of an individual's potential outcomes (POs) under different treatments. We compare three end-to-end learning strategies to overcome this problem.
  arXiv  Detail & Related papers  (2021-06-07T16:30:46Z)
• Loss Bounds for Approximate Influence-Based Abstraction [81.13024471616417]
  Influence-based abstraction aims to gain leverage by modeling local subproblems together with the 'influence' that the rest of the system exerts on them. This paper investigates the performance of such approaches from a theoretical perspective. We show that neural networks trained with cross entropy are well suited to learn approximate influence representations.
  arXiv  Detail & Related papers  (2020-11-03T15:33:10Z)
• Global Context-Aware Progressive Aggregation Network for Salient Object Detection [117.943116761278]
  We propose a novel network named GCPANet to integrate low-level appearance features, high-level semantic features, and global context features. We show that the proposed approach outperforms the state-of-the-art methods both quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2020-03-02T04:26:10Z)
• Nonparametric inference for interventional effects with multiple mediators [0.0]
  We provide theory that allows for more flexible, possibly machine learning-based, estimation techniques. We demonstrate multiple robustness properties of the proposed estimators. Our work thus provides a means of leveraging modern statistical learning techniques in estimation of interventional mediation effects.
  arXiv  Detail & Related papers  (2020-01-16T19:05:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.