From global to local MDI variable importances for random forests and when they are Shapley values

• URL: http://arxiv.org/abs/2111.02218v1
• Date: Wed, 3 Nov 2021 13:38:41 GMT
• Title: From global to local MDI variable importances for random forests and when they are Shapley values
• Authors: Antonio Sutera, Gilles Louppe, Van Anh Huynh-Thu, Louis Wehenkel, Pierre Geurts
• Abstract summary: We first show that the global Mean Decrease of Impurity (MDI) variable importance scores correspond to Shapley values under some conditions. We derive a local MDI importance measure of variable relevance, which has a very natural connection with the global MDI measure and can be related to a new notion of local feature relevance.
• Score: 9.99125500568217
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Random forests have been widely used for their ability to provide so-called importance measures, which give insight at a global (per dataset) level on the relevance of input variables to predict a certain output. On the other hand, methods based on Shapley values have been introduced to refine the analysis of feature relevance in tree-based models to a local (per instance) level. In this context, we first show that the global Mean Decrease of Impurity (MDI) variable importance scores correspond to Shapley values under some conditions. Then, we derive a local MDI importance measure of variable relevance, which has a very natural connection with the global MDI measure and can be related to a new notion of local feature relevance. We further link local MDI importances with Shapley values and discuss them in the light of related measures from the literature. The measures are illustrated through experiments on several classification and regression problems.

Related papers

• Local MDI+: Local Feature Importances for Tree-Based Models [8.532396185972392]
  Local MDI+ (LMDI+) is a novel extension of the MDI+ framework to the sample specific setting.<n>It produces similar instance-level feature importance rankings across multiple random forest fits.<n>It also enables local interpretability use cases, including the identification of closer counterfactuals.
  arXiv  Detail & Related papers  (2025-06-10T15:51:27Z)
• VARSHAP: Addressing Global Dependency Problems in Explainable AI with Variance-Based Local Feature Attribution [3.545940115969205]
  Existing feature attribution methods like SHAP often suffer from global dependence, failing to capture true local model behavior.<n>This paper introduces VARSHAP, a novel model-agnostic local feature attribution method which uses the reduction of prediction variance as the key importance metric of features.
  arXiv  Detail & Related papers  (2025-06-08T17:26:47Z)
• MF-LLM: Simulating Collective Decision Dynamics via a Mean-Field Large Language Model Framework [53.82097200295448]
  Mean-Field LLM (MF-LLM) framework explicitly models the feedback loop between micro-level decisions and macro-level population. MF-LLM alternates between two models: a policy model that generates individual actions based on personal states and group-level information, and a mean field model that updates the population distribution. We evaluate MF-LLM on a real-world social dataset, where it reduces KL divergence to human population distributions by 47 percent over non-mean-field baselines.
  arXiv  Detail & Related papers  (2025-04-30T12:41:51Z)
• ForestLPR: LiDAR Place Recognition in Forests Attentioning Multiple BEV Density Images [38.727720300337296]
  We propose a robust LiDAR-based place recognition method for natural forests, ForestLPR. Cross-sectional images of the forest's geometry at different heights contain the information needed to recognize revisiting a place. Our approach utilizes a visual transformer as the shared backbone to produce sets of local descriptors.
  arXiv  Detail & Related papers  (2025-03-06T14:24:22Z)
• Interpretable Load Forecasting via Representation Learning of Geo-distributed Meteorological Factors [4.998962281945562]
  Meteorological factors (MF) are crucial in day-ahead load forecasting as they significantly influence the electricity consumption behaviors of consumers. The difference in MF collected in various locations within a region may be significant, which poses a challenge in selecting the appropriate MF from numerous locations. A representation learning framework is proposed to extract geo-distributed MF while considering their spatial relationships.
  arXiv  Detail & Related papers  (2025-01-04T09:05:06Z)
• Ranked from Within: Ranking Large Multimodal Models for Visual Question Answering Without Labels [64.94853276821992]
  Large multimodal models (LMMs) are increasingly deployed across diverse applications. Traditional evaluation methods are largely dataset-centric, relying on fixed, labeled datasets and supervised metrics. We explore unsupervised model ranking for LMMs by leveraging their uncertainty signals, such as softmax probabilities.
  arXiv  Detail & Related papers  (2024-12-09T13:05:43Z)
• Model agnostic local variable importance for locally dependent relationships [2.3374134413353254]
  We propose a new model-agnostic method for calculating local variable importance, CLIQUE, that captures locally dependent relationships. We show that CLIQUE emphasizes locally dependent information and properly reduces bias in regions where variables do not affect the response.
  arXiv  Detail & Related papers  (2024-11-13T17:59:44Z)
• Detecting Training Data of Large Language Models via Expectation Maximization [62.28028046993391]
  Membership inference attacks (MIAs) aim to determine whether a specific instance was part of a target model's training data. Applying MIAs to large language models (LLMs) presents unique challenges due to the massive scale of pre-training data and the ambiguous nature of membership. We introduce EM-MIA, a novel MIA method for LLMs that iteratively refines membership scores and prefix scores via an expectation-maximization algorithm.
  arXiv  Detail & Related papers  (2024-10-10T03:31:16Z)
• Data Valuation by Leveraging Global and Local Statistical Information [25.911043100052588]
  We show that both global and local value distributions hold significant potential for data valuation within the context of machine learning. We propose a new data valuation method that estimates Shapley values by incorporating the explored distribution characteristics into an existing method, AME. We also present a new path to address the dynamic data valuation problem by formulating an optimization problem that integrates information of both global and local value distributions.
  arXiv  Detail & Related papers  (2024-05-23T08:58:08Z)
• Adaptive Global-Local Representation Learning and Selection for Cross-Domain Facial Expression Recognition [54.334773598942775]
  Domain shift poses a significant challenge in Cross-Domain Facial Expression Recognition (CD-FER) We propose an Adaptive Global-Local Representation Learning and Selection framework.
  arXiv  Detail & Related papers  (2024-01-20T02:21:41Z)
• MMD-based Variable Importance for Distributional Random Forest [5.0459880125089]
  We introduce a variable importance algorithm for Distributional Random Forests (DRFs) We show that the introduced importance measure is consistent, exhibits high empirical performance on both real and simulated data, and outperforms competitors.
  arXiv  Detail & Related papers  (2023-10-18T17:12:29Z)
• DCID: Deep Canonical Information Decomposition [84.59396326810085]
  We consider the problem of identifying the signal shared between two one-dimensional target variables. We propose ICM, an evaluation metric which can be used in the presence of ground-truth labels. We also propose Deep Canonical Information Decomposition (DCID) - a simple, yet effective approach for learning the shared variables.
  arXiv  Detail & Related papers  (2023-06-27T16:59:06Z)
• Generalizable Metric Network for Cross-domain Person Re-identification [55.71632958027289]
  Cross-domain (i.e., domain generalization) scene presents a challenge in Re-ID tasks. Most existing methods aim to learn domain-invariant or robust features for all domains. We propose a Generalizable Metric Network (GMN) to explore sample similarity in the sample-pair space.
  arXiv  Detail & Related papers  (2023-06-21T03:05:25Z)
• Improving Mutual Information Estimation with Annealed and Energy-Based Bounds [20.940022170594816]
  Mutual information (MI) is a fundamental quantity in information theory and machine learning. We present a unifying view of existing MI bounds from the perspective of importance sampling. We propose three novel bounds based on this approach.
  arXiv  Detail & Related papers  (2023-03-13T10:47:24Z)
• Federated and Generalized Person Re-identification through Domain and Feature Hallucinating [88.77196261300699]
  We study the problem of federated domain generalization (FedDG) for person re-identification (re-ID) We propose a novel method, called "Domain and Feature Hallucinating (DFH)", to produce diverse features for learning generalized local and global models. Our method achieves the state-of-the-art performance for FedDG on four large-scale re-ID benchmarks.
  arXiv  Detail & Related papers  (2022-03-05T09:15:13Z)
• META: Mimicking Embedding via oThers' Aggregation for Generalizable Person Re-identification [68.39849081353704]
  Domain generalizable (DG) person re-identification (ReID) aims to test across unseen domains without access to the target domain data at training time. This paper presents a new approach called Mimicking Embedding via oThers' Aggregation (META) for DG ReID.
  arXiv  Detail & Related papers  (2021-12-16T08:06:50Z)
• Triplot: model agnostic measures and visualisations for variable importance in predictive models that take into account the hierarchical correlation structure [3.0036519884678894]
  We propose new methods to support model analysis by exploiting the information about the correlation between variables. We show how to analyze groups of variables (aspects) both when they are proposed by the user and when they should be determined automatically. We also present the new type of model visualisation, triplot, which exploits a hierarchical structure of variable grouping to produce a high information density model visualisation.
  arXiv  Detail & Related papers  (2021-04-07T21:29:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.