Interpretation and Simplification of Deep Forest

• URL: http://arxiv.org/abs/2001.04721v4
• Date: Sat, 12 Dec 2020 04:35:05 GMT
• Title: Interpretation and Simplification of Deep Forest
• Authors: Sangwon Kim, Mira Jeong, Byoung Chul Ko
• Abstract summary: We consider quantifying the feature contributions and frequency of the fully trained deep RF in the form of a decision rule set. Model simplification is achieved by eliminating unnecessary rules by measuring the feature contributions. Experiment results have shown that a feature contribution analysis allows a black box model to be decomposed for quantitatively interpreting a rule set.
• Score: 4.576379639081977
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper proposes a new method for interpreting and simplifying a black box model of a deep random forest (RF) using a proposed rule elimination. In deep RF, a large number of decision trees are connected to multiple layers, thereby making an analysis difficult. It has a high performance similar to that of a deep neural network (DNN), but achieves a better generalizability. Therefore, in this study, we consider quantifying the feature contributions and frequency of the fully trained deep RF in the form of a decision rule set. The feature contributions provide a basis for determining how features affect the decision process in a rule set. Model simplification is achieved by eliminating unnecessary rules by measuring the feature contributions. Consequently, the simplified model has fewer parameters and rules than before. Experiment results have shown that a feature contribution analysis allows a black box model to be decomposed for quantitatively interpreting a rule set. The proposed method was successfully applied to various deep RF models and benchmark datasets while maintaining a robust performance despite the elimination of a large number of rules.

Related papers

• A Scalable Matrix Visualization for Understanding Tree Ensemble Classifiers [20.416696003269674]
  This paper introduces a scalable visual analysis method to explain tree ensemble classifiers that contain tens of thousands of rules. We develop an anomaly-biased model reduction method to prioritize these rules at each hierarchical level. Our method fosters a deeper understanding of both common and anomalous rules, thereby enhancing interpretability without sacrificing comprehensiveness.
  arXiv  Detail & Related papers  (2024-09-05T01:48:11Z)
• Revisiting SMoE Language Models by Evaluating Inefficiencies with Task Specific Expert Pruning [78.72226641279863]
  Sparse Mixture of Expert (SMoE) models have emerged as a scalable alternative to dense models in language modeling. Our research explores task-specific model pruning to inform decisions about designing SMoE architectures. We introduce an adaptive task-aware pruning technique UNCURL to reduce the number of experts per MoE layer in an offline manner post-training.
  arXiv  Detail & Related papers  (2024-09-02T22:35:03Z)
• A survey and taxonomy of methods interpreting random forest models [0.0]
  The interpretability of random forest (RF) models is a research topic of growing interest in the machine learning (ML) community. RF resulting model is regarded as a "black box" because of its numerous deep decision trees. This paper aims to provide an extensive review of methods used in the literature to interpret RF resulting models.
  arXiv  Detail & Related papers  (2024-07-17T17:33:32Z)
• Forest-ORE: Mining Optimal Rule Ensemble to interpret Random Forest models [0.0]
  We present Forest-ORE, a method that makes Random Forest (RF) interpretable via an optimized rule ensemble (ORE) for local and global interpretation. A comparative analysis of well-known methods shows that Forest-ORE provides an excellent trade-off between predictive performance, interpretability coverage, and model size.
  arXiv  Detail & Related papers  (2024-03-26T10:54:07Z)
• Deep Learning for the Benes Filter [91.3755431537592]
  We present a new numerical method based on the mesh-free neural network representation of the density of the solution of the Benes model. We discuss the role of nonlinearity in the filtering model equations for the choice of the domain of the neural network.
  arXiv  Detail & Related papers  (2022-03-09T14:08:38Z)
• Interpreting Deep Learning Model Using Rule-based Method [36.01435823818395]
  We propose a multi-level decision framework to provide comprehensive interpretation for the deep neural network model. By fitting decision trees for each neuron and aggregate them together, a multi-level decision structure (MLD) is constructed at first. Experiments on the MNIST and National Free Pre-Pregnancy Check-up dataset are carried out to demonstrate the effectiveness and interpretability of MLD framework.
  arXiv  Detail & Related papers  (2020-10-15T15:30:00Z)
• Diverse Rule Sets [20.170305081348328]
  Rule-based systems are experiencing a renaissance owing to their intuitive if-then representation. We propose a novel approach of inferring diverse rule sets, by optimizing small overlap among decision rules. We then devise an efficient randomized algorithm, which samples rules that are highly discriminative and have small overlap.
  arXiv  Detail & Related papers  (2020-06-17T14:15:25Z)
• Explainable Matrix -- Visualization for Global and Local Interpretability of Random Forest Classification Ensembles [78.6363825307044]
  We propose Explainable Matrix (ExMatrix), a novel visualization method for Random Forest (RF) interpretability. It employs a simple yet powerful matrix-like visual metaphor, where rows are rules, columns are features, and cells are rules predicates. ExMatrix applicability is confirmed via different examples, showing how it can be used in practice to promote RF models interpretability.
  arXiv  Detail & Related papers  (2020-05-08T21:03:48Z)
• Lower bounds in multiple testing: A framework based on derandomized proxies [107.69746750639584]
  This paper introduces an analysis strategy based on derandomization, illustrated by applications to various concrete models. We provide numerical simulations of some of these lower bounds, and show a close relation to the actual performance of the Benjamini-Hochberg (BH) algorithm.
  arXiv  Detail & Related papers  (2020-05-07T19:59:51Z)
• Deep Unfolding Network for Image Super-Resolution [159.50726840791697]
  This paper proposes an end-to-end trainable unfolding network which leverages both learning-based methods and model-based methods. The proposed network inherits the flexibility of model-based methods to super-resolve blurry, noisy images for different scale factors via a single model.
  arXiv  Detail & Related papers  (2020-03-23T17:55:42Z)
• Beyond Dropout: Feature Map Distortion to Regularize Deep Neural Networks [107.77595511218429]
  In this paper, we investigate the empirical Rademacher complexity related to intermediate layers of deep neural networks. We propose a feature distortion method (Disout) for addressing the aforementioned problem. The superiority of the proposed feature map distortion for producing deep neural network with higher testing performance is analyzed and demonstrated.
  arXiv  Detail & Related papers  (2020-02-23T13:59:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.