Interpretation and Simplification of Deep Forest
- URL: http://arxiv.org/abs/2001.04721v4
- Date: Sat, 12 Dec 2020 04:35:05 GMT
- Title: Interpretation and Simplification of Deep Forest
- Authors: Sangwon Kim, Mira Jeong, Byoung Chul Ko
- Abstract summary: We consider quantifying the feature contributions and frequency of the fully trained deep RF in the form of a decision rule set.
Model simplification is achieved by eliminating unnecessary rules by measuring the feature contributions.
Experiment results have shown that a feature contribution analysis allows a black box model to be decomposed for quantitatively interpreting a rule set.
- Score: 4.576379639081977
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: This paper proposes a new method for interpreting and simplifying a black box
model of a deep random forest (RF) using a proposed rule elimination. In deep
RF, a large number of decision trees are connected to multiple layers, thereby
making an analysis difficult. It has a high performance similar to that of a
deep neural network (DNN), but achieves a better generalizability. Therefore,
in this study, we consider quantifying the feature contributions and frequency
of the fully trained deep RF in the form of a decision rule set. The feature
contributions provide a basis for determining how features affect the decision
process in a rule set. Model simplification is achieved by eliminating
unnecessary rules by measuring the feature contributions. Consequently, the
simplified model has fewer parameters and rules than before. Experiment results
have shown that a feature contribution analysis allows a black box model to be
decomposed for quantitatively interpreting a rule set. The proposed method was
successfully applied to various deep RF models and benchmark datasets while
maintaining a robust performance despite the elimination of a large number of
rules.
Related papers
- A survey and taxonomy of methods interpreting random forest models [0.0]
The interpretability of random forest (RF) models is a research topic of growing interest in the machine learning (ML) community.
RF resulting model is regarded as a "black box" because of its numerous deep decision trees.
This paper aims to provide an extensive review of methods used in the literature to interpret RF resulting models.
arXiv Detail & Related papers (2024-07-17T17:33:32Z) - Forest-ORE: Mining Optimal Rule Ensemble to interpret Random Forest models [0.0]
We present Forest-ORE, a method that makes Random Forest (RF) interpretable via an optimized rule ensemble (ORE) for local and global interpretation.
A comparative analysis of well-known methods shows that Forest-ORE provides an excellent trade-off between predictive performance, interpretability coverage, and model size.
arXiv Detail & Related papers (2024-03-26T10:54:07Z) - Domain Generalization via Rationale Invariance [70.32415695574555]
This paper offers a new perspective to ease the challenge of domain generalization, which involves maintaining robust results even in unseen environments.
We propose treating the element-wise contributions to the final results as the rationale for making a decision and representing the rationale for each sample as a matrix.
Our experiments demonstrate that the proposed approach achieves competitive results across various datasets, despite its simplicity.
arXiv Detail & Related papers (2023-08-22T03:31:40Z) - Deep Learning for the Benes Filter [91.3755431537592]
We present a new numerical method based on the mesh-free neural network representation of the density of the solution of the Benes model.
We discuss the role of nonlinearity in the filtering model equations for the choice of the domain of the neural network.
arXiv Detail & Related papers (2022-03-09T14:08:38Z) - Integrating Regular Expressions with Neural Networks via DFA [40.09868407372605]
It is very important to integrate the rule knowledge into neural networks to build a hybrid model that achieves better performance.
Specifically, the human-designed rules are formulated as Regular Expressions (REs)
We propose to use the MDFA as an intermediate model to capture the matched RE patterns as rule-based features for each input sentence.
arXiv Detail & Related papers (2021-09-07T05:48:51Z) - Interpreting Deep Learning Model Using Rule-based Method [36.01435823818395]
We propose a multi-level decision framework to provide comprehensive interpretation for the deep neural network model.
By fitting decision trees for each neuron and aggregate them together, a multi-level decision structure (MLD) is constructed at first.
Experiments on the MNIST and National Free Pre-Pregnancy Check-up dataset are carried out to demonstrate the effectiveness and interpretability of MLD framework.
arXiv Detail & Related papers (2020-10-15T15:30:00Z) - Diverse Rule Sets [20.170305081348328]
Rule-based systems are experiencing a renaissance owing to their intuitive if-then representation.
We propose a novel approach of inferring diverse rule sets, by optimizing small overlap among decision rules.
We then devise an efficient randomized algorithm, which samples rules that are highly discriminative and have small overlap.
arXiv Detail & Related papers (2020-06-17T14:15:25Z) - Explainable Matrix -- Visualization for Global and Local
Interpretability of Random Forest Classification Ensembles [78.6363825307044]
We propose Explainable Matrix (ExMatrix), a novel visualization method for Random Forest (RF) interpretability.
It employs a simple yet powerful matrix-like visual metaphor, where rows are rules, columns are features, and cells are rules predicates.
ExMatrix applicability is confirmed via different examples, showing how it can be used in practice to promote RF models interpretability.
arXiv Detail & Related papers (2020-05-08T21:03:48Z) - Lower bounds in multiple testing: A framework based on derandomized
proxies [107.69746750639584]
This paper introduces an analysis strategy based on derandomization, illustrated by applications to various concrete models.
We provide numerical simulations of some of these lower bounds, and show a close relation to the actual performance of the Benjamini-Hochberg (BH) algorithm.
arXiv Detail & Related papers (2020-05-07T19:59:51Z) - Deep Unfolding Network for Image Super-Resolution [159.50726840791697]
This paper proposes an end-to-end trainable unfolding network which leverages both learning-based methods and model-based methods.
The proposed network inherits the flexibility of model-based methods to super-resolve blurry, noisy images for different scale factors via a single model.
arXiv Detail & Related papers (2020-03-23T17:55:42Z) - Beyond Dropout: Feature Map Distortion to Regularize Deep Neural
Networks [107.77595511218429]
In this paper, we investigate the empirical Rademacher complexity related to intermediate layers of deep neural networks.
We propose a feature distortion method (Disout) for addressing the aforementioned problem.
The superiority of the proposed feature map distortion for producing deep neural network with higher testing performance is analyzed and demonstrated.
arXiv Detail & Related papers (2020-02-23T13:59:13Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.