Provable Recovery of Locally Important Signed Features and Interactions from Random Forest

• URL: http://arxiv.org/abs/2512.11081v1
• Date: Thu, 11 Dec 2025 19:53:15 GMT
• Title: Provable Recovery of Locally Important Signed Features and Interactions from Random Forest
• Abstract summary: In many domains, such as personalized medicine, local interpretations for individual predictions are often required.<n>We propose a novel, local, model-specific FII method that identifies frequent co-occurrences of features along decision paths.<n>We prove that our method consistently recovers the true local signal features and their interactions under a Locally Spike Sparse (LSS) model.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Feature and Interaction Importance (FII) methods are essential in supervised learning for assessing the relevance of input variables and their interactions in complex prediction models. In many domains, such as personalized medicine, local interpretations for individual predictions are often required, rather than global scores summarizing overall feature importance. Random Forests (RFs) are widely used in these settings, and existing interpretability methods typically exploit tree structures and split statistics to provide model-specific insights. However, theoretical understanding of local FII methods for RF remains limited, making it unclear how to interpret high importance scores for individual predictions. We propose a novel, local, model-specific FII method that identifies frequent co-occurrences of features along decision paths, combining global patterns with those observed on paths specific to a given test point. We prove that our method consistently recovers the true local signal features and their interactions under a Locally Spike Sparse (LSS) model and also identifies whether large or small feature values drive a prediction. We illustrate the usefulness of our method and theoretical results through simulation studies and a real-world data example.

Related papers

• xplainfi: Feature Importance and Statistical Inference for Machine Learning in R [12.35431733807122]
  We introduce xplainfi, an R package built on top of the mlr3 ecosystem for global, loss-based feature importance methods for machine learning models.<n>xplainfi produces importance scores consistent with existing implementations across multiple simulation settings and learner types.
  arXiv  Detail & Related papers  (2026-03-16T14:04:48Z)
• General OOD Detection via Model-aware and Subspace-aware Variable Priority [1.293838707857305]
  Out-of-distribution (OOD) detection is essential for determining when a supervised model encounters inputs that differ meaningfully from its training distribution.<n>We introduce a framework for OOD detection that is simultaneously model aware and subspace aware, and that embeds variable prioritization directly into the detection step.
  arXiv  Detail & Related papers  (2025-12-15T05:55:35Z)
• Interpretable Network-assisted Random Forest+ [2.776181981917578]
  We develop a family of flexible network-assisted models built upon a generalization of random forests (RF+)<n>We provide both global and local importance measures as well as sample influence measures to assess the impact of a given observation.<n>This suite of tools broadens the scope and applicability of network-assisted machine learning for high-impact problems.
  arXiv  Detail & Related papers  (2025-09-19T05:22:06Z)
• Local vs. Global Models for Hierarchical Forecasting [0.0]
  This study explores the influence of distinct information utilisation on the accuracy of hierarchical forecasts. We develop Global Forecasting Models (GFMs) to exploit cross-series and cross-hierarchies information. Two specific GFMs based on LightGBM are introduced, demonstrating superior accuracy and lower model complexity.
  arXiv  Detail & Related papers  (2024-11-10T08:51:49Z)
• Testing Generalizability in Causal Inference [3.547529079746247]
  No formal procedure exists for statistically evaluating generalizability in machine learning algorithms.<n>We propose a systematic framework for statistically evaluating the generalizability of high-dimensional causal inference models.
  arXiv  Detail & Related papers  (2024-11-05T11:44:00Z)
• Adaptive Global-Local Representation Learning and Selection for Cross-Domain Facial Expression Recognition [54.334773598942775]
  Domain shift poses a significant challenge in Cross-Domain Facial Expression Recognition (CD-FER) We propose an Adaptive Global-Local Representation Learning and Selection framework.
  arXiv  Detail & Related papers  (2024-01-20T02:21:41Z)
• Think Twice Before Selection: Federated Evidential Active Learning for Medical Image Analysis with Domain Shifts [11.562953837452126]
  We make the first attempt to assess the informativeness of local data derived from diverse domains. We propose a novel methodology termed Federated Evidential Active Learning (FEAL) to calibrate the data evaluation under domain shift.
  arXiv  Detail & Related papers  (2023-12-05T08:32:27Z)
• Numerically assisted determination of local models in network scenarios [55.2480439325792]
  We develop a numerical tool for finding explicit local models that reproduce a given statistical behaviour. We provide conjectures for the critical visibilities of the Greenberger-Horne-Zeilinger (GHZ) and W distributions. The developed codes and documentation are publicly available at281.com/mariofilho/localmodels.
  arXiv  Detail & Related papers  (2023-03-17T13:24:04Z)
• Monitoring Shortcut Learning using Mutual Information [16.17600110257266]
  Shortcut learning is evaluated on real-world data that does not contain spurious correlations. Experiments demonstrate that MI can be used as a metric network shortcut network.
  arXiv  Detail & Related papers  (2022-06-27T03:55:23Z)
• DRFLM: Distributionally Robust Federated Learning with Inter-client Noise via Local Mixup [58.894901088797376]
  federated learning has emerged as a promising approach for training a global model using data from multiple organizations without leaking their raw data. We propose a general framework to solve the above two challenges simultaneously. We provide comprehensive theoretical analysis including robustness analysis, convergence analysis, and generalization ability.
  arXiv  Detail & Related papers  (2022-04-16T08:08:29Z)
• Federated and Generalized Person Re-identification through Domain and Feature Hallucinating [88.77196261300699]
  We study the problem of federated domain generalization (FedDG) for person re-identification (re-ID) We propose a novel method, called "Domain and Feature Hallucinating (DFH)", to produce diverse features for learning generalized local and global models. Our method achieves the state-of-the-art performance for FedDG on four large-scale re-ID benchmarks.
  arXiv  Detail & Related papers  (2022-03-05T09:15:13Z)
• Learning Bias-Invariant Representation by Cross-Sample Mutual Information Minimization [77.8735802150511]
  We propose a cross-sample adversarial debiasing (CSAD) method to remove the bias information misused by the target task. The correlation measurement plays a critical role in adversarial debiasing and is conducted by a cross-sample neural mutual information estimator. We conduct thorough experiments on publicly available datasets to validate the advantages of the proposed method over state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-08-11T21:17:02Z)
• Counterfactual Maximum Likelihood Estimation for Training Deep Networks [83.44219640437657]
  Deep learning models are prone to learning spurious correlations that should not be learned as predictive clues. We propose a causality-based training framework to reduce the spurious correlations caused by observable confounders. We conduct experiments on two real-world tasks: Natural Language Inference (NLI) and Image Captioning.
  arXiv  Detail & Related papers  (2021-06-07T17:47:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.