Interpretable Machine Learning with an Ensemble of Gradient Boosting Machines

• URL: http://arxiv.org/abs/2010.07388v1
• Date: Wed, 14 Oct 2020 20:18:40 GMT
• Title: Interpretable Machine Learning with an Ensemble of Gradient Boosting Machines
• Authors: Andrei V. Konstantinov and Lev V. Utkin
• Abstract summary: Method is based on using an ensemble of gradient boosting machines (GBMs) A lot of numerical experiments with an algorithm implementing the proposed method on synthetic and real datasets demonstrate its efficiency and properties for local and global interpretation.
• Score: 5.482532589225552
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A method for the local and global interpretation of a black-box model on the basis of the well-known generalized additive models is proposed. It can be viewed as an extension or a modification of the algorithm using the neural additive model. The method is based on using an ensemble of gradient boosting machines (GBMs) such that each GBM is learned on a single feature and produces a shape function of the feature. The ensemble is composed as a weighted sum of separate GBMs resulting a weighted sum of shape functions which form the generalized additive model. GBMs are built in parallel using randomized decision trees of depth 1, which provide a very simple architecture. Weights of GBMs as well as features are computed in each iteration of boosting by using the Lasso method and then updated by means of a specific smoothing procedure. In contrast to the neural additive model, the method provides weights of features in the explicit form, and it is simply trained. A lot of numerical experiments with an algorithm implementing the proposed method on synthetic and real datasets demonstrate its efficiency and properties for local and global interpretation.

Related papers

• Universal Neural Functionals [67.80283995795985]
  A challenging problem in many modern machine learning tasks is to process weight-space features. Recent works have developed promising weight-space models that are equivariant to the permutation symmetries of simple feedforward networks. This work proposes an algorithm that automatically constructs permutation equivariant models for any weight space.
  arXiv  Detail & Related papers  (2024-02-07T20:12:27Z)
• Dual feature-based and example-based explanation methods [2.024925013349319]
  A new approach to the local and global explanation is proposed. It is based on selecting a convex hull constructed for the finite number of points around an explained instance. The code of proposed algorithms is available.
  arXiv  Detail & Related papers  (2024-01-29T16:53:04Z)
• Computing SHAP Efficiently Using Model Structure Information [3.6626323701161665]
  We propose methods that compute SHAP exactly in time or even faster for SHAP definitions that satisfy our additivity and dummy assumptions. For the first case, we demonstrate an additive property and a way to compute SHAP from the lower-order functional components. For the second case, we derive formulas that can compute SHAP in time. Both methods yield exact SHAP results.
  arXiv  Detail & Related papers  (2023-09-05T17:48:09Z)
• Generalizing Backpropagation for Gradient-Based Interpretability [103.2998254573497]
  We show that the gradient of a model is a special case of a more general formulation using semirings. This observation allows us to generalize the backpropagation algorithm to efficiently compute other interpretable statistics.
  arXiv  Detail & Related papers  (2023-07-06T15:19:53Z)
• Interpretable Ensembles of Hyper-Rectangles as Base Models [5.482532589225552]
  A new ensemble-based model with uniformly generated axis-parallel hyper-rectangles as base models (HRBM) is proposed. It is proposed to incorporate HRBMs into the gradient boosting machine (GBM)
  arXiv  Detail & Related papers  (2023-03-15T13:50:36Z)
• Symbolic Metamodels for Interpreting Black-boxes Using Primitive Functions [15.727276506140878]
  One approach for interpreting black-box machine learning models is to find a global approximation of the model using simple interpretable functions. In this work, we propose a new method for finding interpretable metamodels.
  arXiv  Detail & Related papers  (2023-02-09T17:30:43Z)
• Feature Weighted Non-negative Matrix Factorization [92.45013716097753]
  We propose the Feature weighted Non-negative Matrix Factorization (FNMF) in this paper. FNMF learns the weights of features adaptively according to their importances. It can be solved efficiently with the suggested optimization algorithm.
  arXiv  Detail & Related papers  (2021-03-24T21:17:17Z)
• Model Fusion with Kullback--Leibler Divergence [58.20269014662046]
  We propose a method to fuse posterior distributions learned from heterogeneous datasets. Our algorithm relies on a mean field assumption for both the fused model and the individual dataset posteriors.
  arXiv  Detail & Related papers  (2020-07-13T03:27:45Z)
• The data-driven physical-based equations discovery using evolutionary approach [77.34726150561087]
  We describe the algorithm for the mathematical equations discovery from the given observations data. The algorithm combines genetic programming with the sparse regression. It could be used for governing analytical equation discovery as well as for partial differential equations (PDE) discovery.
  arXiv  Detail & Related papers  (2020-04-03T17:21:57Z)
• Particle-Gibbs Sampling For Bayesian Feature Allocation Models [77.57285768500225]
  Most widely used MCMC strategies rely on an element wise Gibbs update of the feature allocation matrix. We have developed a Gibbs sampler that can update an entire row of the feature allocation matrix in a single move. This sampler is impractical for models with a large number of features as the computational complexity scales exponentially in the number of features. We develop a Particle Gibbs sampler that targets the same distribution as the row wise Gibbs updates, but has computational complexity that only grows linearly in the number of features.
  arXiv  Detail & Related papers  (2020-01-25T22:11:51Z)
• Analysis of Bayesian Inference Algorithms by the Dynamical Functional Approach [2.8021833233819486]
  We analyze an algorithm for approximate inference with large Gaussian latent variable models in a student-trivial scenario. For the case of perfect data-model matching, the knowledge of static order parameters derived from the replica method allows us to obtain efficient algorithmic updates.
  arXiv  Detail & Related papers  (2020-01-14T17:22:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.