Mixture of Decision Trees for Interpretable Machine Learning
- URL: http://arxiv.org/abs/2211.14617v1
- Date: Sat, 26 Nov 2022 17:09:51 GMT
- Title: Mixture of Decision Trees for Interpretable Machine Learning
- Authors: Simeon Br\"uggenj\"urgen, Nina Schaaf, Pascal Kerschke, Marco F. Huber
- Abstract summary: This work introduces a novel interpretable machine learning method called Mixture of Decision Trees (MoDT)
MoDT can be considered as a method that improves performance while maintaining interpretability by making each of its decisions understandable and traceable to humans.
- Score: 4.180840853105103
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: This work introduces a novel interpretable machine learning method called
Mixture of Decision Trees (MoDT). It constitutes a special case of the Mixture
of Experts ensemble architecture, which utilizes a linear model as gating
function and decision trees as experts. Our proposed method is ideally suited
for problems that cannot be satisfactorily learned by a single decision tree,
but which can alternatively be divided into subproblems. Each subproblem can
then be learned well from a single decision tree. Therefore, MoDT can be
considered as a method that improves performance while maintaining
interpretability by making each of its decisions understandable and traceable
to humans.
Our work is accompanied by a Python implementation, which uses an
interpretable gating function, a fast learning algorithm, and a direct
interface to fine-tuned interpretable visualization methods. The experiments
confirm that the implementation works and, more importantly, show the
superiority of our approach compared to single decision trees and random
forests of similar complexity.
Related papers
- A Unified Approach to Extract Interpretable Rules from Tree Ensembles via Integer Programming [2.1408617023874443]
Tree ensemble methods are known for their effectiveness in supervised classification and regression tasks.
Our work aims to extract an optimized list of rules from a trained tree ensemble, providing the user with a condensed, interpretable model.
arXiv Detail & Related papers (2024-06-30T22:33:47Z) - Learning accurate and interpretable decision trees [27.203303726977616]
We develop approaches to design decision tree learning algorithms given repeated access to data from the same domain.
We study the sample complexity of tuning prior parameters in Bayesian decision tree learning, and extend our results to decision tree regression.
We also study the interpretability of the learned decision trees and introduce a data-driven approach for optimizing the explainability versus accuracy trade-off using decision trees.
arXiv Detail & Related papers (2024-05-24T20:10:10Z) - Greedy Algorithm for Inference of Decision Trees from Decision Rule
Systems [0.0]
Decision trees and decision rule systems play important roles as attributes, knowledge representation tools, and algorithms.
In this paper, we consider the inverse transformation problem, which is not so simple.
Instead of constructing an entire decision tree, our study focuses on a greedy time algorithm that simulates the operation of a decision tree on a given attribute.
arXiv Detail & Related papers (2024-01-08T09:28:55Z) - Interpretable Decision Tree Search as a Markov Decision Process [8.530182510074983]
Finding an optimal decision tree for a supervised learning task is a challenging problem to solve at scale.
It was recently proposed to frame the problem as a Markov Decision Problem (MDP) and use deep reinforcement learning to tackle scaling.
We propose instead to scale the resolution of such MDPs using an information-theoretic tests generating function that generates for every state.
arXiv Detail & Related papers (2023-09-22T08:18:08Z) - TreeDQN: Learning to minimize Branch-and-Bound tree [78.52895577861327]
Branch-and-Bound is a convenient approach to solving optimization tasks in the form of Mixed Linear Programs.
The efficiency of the solver depends on the branchning used to select a variable for splitting.
We propose a reinforcement learning method that can efficiently learn the branching.
arXiv Detail & Related papers (2023-06-09T14:01:26Z) - Towards Improved and Interpretable Deep Metric Learning via Attentive
Grouping [103.71992720794421]
Grouping has been commonly used in deep metric learning for computing diverse features.
We propose an improved and interpretable grouping method to be integrated flexibly with any metric learning framework.
arXiv Detail & Related papers (2020-11-17T19:08:24Z) - Rectified Decision Trees: Exploring the Landscape of Interpretable and
Effective Machine Learning [66.01622034708319]
We propose a knowledge distillation based decision trees extension, dubbed rectified decision trees (ReDT)
We extend the splitting criteria and the ending condition of the standard decision trees, which allows training with soft labels.
We then train the ReDT based on the soft label distilled from a well-trained teacher model through a novel jackknife-based method.
arXiv Detail & Related papers (2020-08-21T10:45:25Z) - MurTree: Optimal Classification Trees via Dynamic Programming and Search [61.817059565926336]
We present a novel algorithm for learning optimal classification trees based on dynamic programming and search.
Our approach uses only a fraction of the time required by the state-of-the-art and can handle datasets with tens of thousands of instances.
arXiv Detail & Related papers (2020-07-24T17:06:55Z) - Learning with Differentiable Perturbed Optimizers [54.351317101356614]
We propose a systematic method to transform operations into operations that are differentiable and never locally constant.
Our approach relies on perturbeds, and can be used readily together with existing solvers.
We show how this framework can be connected to a family of losses developed in structured prediction, and give theoretical guarantees for their use in learning tasks.
arXiv Detail & Related papers (2020-02-20T11:11:32Z) - Parameterizing Branch-and-Bound Search Trees to Learn Branching Policies [76.83991682238666]
Branch and Bound (B&B) is the exact tree search method typically used to solve Mixed-Integer Linear Programming problems (MILPs)
We propose a novel imitation learning framework, and introduce new input features and architectures to represent branching.
arXiv Detail & Related papers (2020-02-12T17:43:23Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.