A Generalized Variable Importance Metric and Estimator for Black Box Machine Learning Models

• URL: http://arxiv.org/abs/2212.09931v3
• Date: Sat, 23 Dec 2023 21:41:07 GMT
• Title: A Generalized Variable Importance Metric and Estimator for Black Box Machine Learning Models
• Authors: Mohammad Kaviul Anam Khan, Olli Saarela and Rafal Kustra
• Abstract summary: We define a population parameter, Generalized Variable Importance Metric (GVIM)'', to measure importance of predictors for black box machine learning methods. We extend previously published results to show that the defined GVIM can be represented as a function of the Conditional Average Treatment Effect (CATE) for any kind of a predictor.
• Score: 0.21249247666376617
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper we define a population parameter, ``Generalized Variable Importance Metric (GVIM)'', to measure importance of predictors for black box machine learning methods, where the importance is not represented by model-based parameter. GVIM is defined for each input variable, using the true conditional expectation function, and it measures the variable's importance in affecting a continuous or a binary response. We extend previously published results to show that the defined GVIM can be represented as a function of the Conditional Average Treatment Effect (CATE) for any kind of a predictor, which gives it a causal interpretation and further justification as an alternative to classical measures of significance that are only available in simple parametric models. Extensive set of simulations using realistically complex relationships between covariates and outcomes and number of regression techniques of varying degree of complexity show the performance of our proposed estimator of the GVIM.

Related papers

• Measuring Variable Importance in Individual Treatment Effect Estimation with High Dimensional Data [35.104681814241104]
  Causal machine learning (ML) promises to provide powerful tools for estimating individual treatment effects. ML methods still face the significant challenge of interpretability, which is crucial for medical applications. We propose a new algorithm based on the Conditional Permutation Importance (CPI) method for statistically rigorous variable importance assessment.
  arXiv  Detail & Related papers  (2024-08-23T11:44:07Z)
• Overparameterized Multiple Linear Regression as Hyper-Curve Fitting [0.0]
  It is proven that a linear model will produce exact predictions even in the presence of nonlinear dependencies that violate the model assumptions. The hyper-curve approach is especially suited for the regularization of problems with noise in predictor variables and can be used to remove noisy and "improper" predictors from the model.
  arXiv  Detail & Related papers  (2024-04-11T15:43:11Z)
• Towards Generalizable and Interpretable Motion Prediction: A Deep Variational Bayes Approach [54.429396802848224]
  This paper proposes an interpretable generative model for motion prediction with robust generalizability to out-of-distribution cases. For interpretability, the model achieves the target-driven motion prediction by estimating the spatial distribution of long-term destinations. Experiments on motion prediction datasets validate that the fitted model can be interpretable and generalizable.
  arXiv  Detail & Related papers  (2024-03-10T04:16:04Z)
• Partially factorized variational inference for high-dimensional mixed models [0.0]
  Variational inference (VI) methods are a popular way to perform such computations. We show that standard VI (i.e. mean-field) dramatically underestimates posterior uncertainty in high-dimensions. We then show how appropriately relaxing the mean-field assumption leads to VI methods whose uncertainty quantification does not deteriorate in high-dimensions.
  arXiv  Detail & Related papers  (2023-12-20T16:12:37Z)
• Generalizing Backpropagation for Gradient-Based Interpretability [103.2998254573497]
  We show that the gradient of a model is a special case of a more general formulation using semirings. This observation allows us to generalize the backpropagation algorithm to efficiently compute other interpretable statistics.
  arXiv  Detail & Related papers  (2023-07-06T15:19:53Z)
• Variable Importance Matching for Causal Inference [73.25504313552516]
  We describe a general framework called Model-to-Match that achieves these goals. Model-to-Match uses variable importance measurements to construct a distance metric. We operationalize the Model-to-Match framework with LASSO.
  arXiv  Detail & Related papers  (2023-02-23T00:43:03Z)
• ER: Equivariance Regularizer for Knowledge Graph Completion [107.51609402963072]
  We propose a new regularizer, namely, Equivariance Regularizer (ER) ER can enhance the generalization ability of the model by employing the semantic equivariance between the head and tail entities. The experimental results indicate a clear and substantial improvement over the state-of-the-art relation prediction methods.
  arXiv  Detail & Related papers  (2022-06-24T08:18:05Z)
• Functional Mixtures-of-Experts [0.24578723416255746]
  We consider the statistical analysis of heterogeneous data for prediction in situations where the observations include functions. We first present a new family of ME models, named functional ME (FME) in which the predictors are potentially noisy observations. We develop dedicated expectation--maximization algorithms for Lasso-like (EM-Lasso) regularized maximum-likelihood parameter estimation strategies to fit the models.
  arXiv  Detail & Related papers  (2022-02-04T17:32:28Z)
• Trustworthy Multimodal Regression with Mixture of Normal-inverse Gamma Distributions [91.63716984911278]
  We introduce a novel Mixture of Normal-Inverse Gamma distributions (MoNIG) algorithm, which efficiently estimates uncertainty in principle for adaptive integration of different modalities and produces a trustworthy regression result. Experimental results on both synthetic and different real-world data demonstrate the effectiveness and trustworthiness of our method on various multimodal regression tasks.
  arXiv  Detail & Related papers  (2021-11-11T14:28:12Z)
• Identifiable Energy-based Representations: An Application to Estimating Heterogeneous Causal Effects [83.66276516095665]
  Conditional average treatment effects (CATEs) allow us to understand the effect heterogeneity across a large population of individuals. Typical CATE learners assume all confounding variables are measured in order for the CATE to be identifiable. We propose an energy-based model (EBM) that learns a low-dimensional representation of the variables by employing a noise contrastive loss function.
  arXiv  Detail & Related papers  (2021-08-06T10:39:49Z)
• Gaussian Process Regression with Local Explanation [28.90948136731314]
  We propose GPR with local explanation, which reveals the feature contributions to the prediction of each sample. In the proposed model, both the prediction and explanation for each sample are performed using an easy-to-interpret locally linear model. For a new test sample, the proposed model can predict the values of its target variable and weight vector, as well as their uncertainties.
  arXiv  Detail & Related papers  (2020-07-03T13:22:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.