Cluster-Based Random Forest Visualization and Interpretation

• URL: http://arxiv.org/abs/2507.22665v1
• Date: Wed, 30 Jul 2025 13:22:28 GMT
• Title: Cluster-Based Random Forest Visualization and Interpretation
• Authors: Max Sondag, Christofer Meinecke, Dennis Collaris, Tatiana von Landesberger, Stef van den Elzen,
• Abstract summary: This paper presents a visualization method and system to increase interpretability of random forests.<n>We cluster similar trees which enables users to interpret how the model performs in general.<n>We also propose two new visualization methods that visualize both clustered and individual decision trees.
• Score: 3.3362962017403364
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Random forests are a machine learning method used to automatically classify datasets and consist of a multitude of decision trees. While these random forests often have higher performance and generalize better than a single decision tree, they are also harder to interpret. This paper presents a visualization method and system to increase interpretability of random forests. We cluster similar trees which enables users to interpret how the model performs in general without needing to analyze each individual decision tree in detail, or interpret an oversimplified summary of the full forest. To meaningfully cluster the decision trees, we introduce a new distance metric that takes into account both the decision rules as well as the predictions of a pair of decision trees. We also propose two new visualization methods that visualize both clustered and individual decision trees: (1) The Feature Plot, which visualizes the topological position of features in the decision trees, and (2) the Rule Plot, which visualizes the decision rules of the decision trees. We demonstrate the efficacy of our approach through a case study on the "Glass" dataset, which is a relatively complex standard machine learning dataset, as well as a small user study.

Related papers

• Learning Decision Trees as Amortized Structure Inference [59.65621207449269]
  We propose a hybrid amortized structure inference approach to learn predictive decision tree ensembles given data.<n>We show that our approach, DT-GFN, outperforms state-of-the-art decision tree and deep learning methods on standard classification benchmarks.
  arXiv  Detail & Related papers  (2025-03-10T07:05:07Z)
• Learning accurate and interpretable tree-based models [27.203303726977616]
  We develop approaches to design tree-based learning algorithms given repeated access to data from the same domain.<n>We propose novel parameterized classes of node splitting criteria in top-down algorithms, which interpolate between popularly used entropy and Gini impurity based criteria.<n>We extend our results to tuning popular tree-based ensembles, including random forests and gradient-boosted trees.
  arXiv  Detail & Related papers  (2024-05-24T20:10:10Z)
• Learning a Decision Tree Algorithm with Transformers [75.96920867382859]
  We introduce MetaTree, a transformer-based model trained via meta-learning to directly produce strong decision trees. We fit both greedy decision trees and globally optimized decision trees on a large number of datasets, and train MetaTree to produce only the trees that achieve strong generalization performance.
  arXiv  Detail & Related papers  (2024-02-06T07:40:53Z)
• ViTree: Single-path Neural Tree for Step-wise Interpretable Fine-grained Visual Categorization [56.37520969273242]
  We introduce ViTree, a novel approach for fine-grained visual categorization. By traversing the tree paths, ViTree effectively selects patches from transformer-processed features to highlight informative local regions. This patch and path selectivity enhances model interpretability of ViTree, enabling better insights into the model's inner workings.
  arXiv  Detail & Related papers  (2024-01-30T14:32:25Z)
• Theoretical and Empirical Advances in Forest Pruning [0.0]
  We revisit forest pruning, an approach that aims to have the best of both worlds: the accuracy of regression forests and the interpretability of regression trees.<n>We prove the advantage of a Lasso-pruned forest over its unpruned counterpart under weak assumptions.<n>We test the accuracy of pruned regression forests against their unpruned counterparts on 19 different datasets.
  arXiv  Detail & Related papers  (2024-01-10T20:02:47Z)
• Construction of Decision Trees and Acyclic Decision Graphs from Decision Rule Systems [0.0]
  We study the complexity of constructing decision trees and acyclic decision graphs representing decision trees from decision rule systems. We discuss the possibility of not building the entire decision tree, but describing the computation path in this tree for the given input.
  arXiv  Detail & Related papers  (2023-05-02T18:40:48Z)
• Unboxing Tree Ensembles for interpretability: a hierarchical visualization tool and a multivariate optimal re-built tree [0.34530027457862006]
  We develop an interpretable representation of a tree-ensemble model that can provide valuable insights into its behavior. The proposed model is effective in yielding a shallow interpretable tree approxing the tree-ensemble decision function.
  arXiv  Detail & Related papers  (2023-02-15T10:43:31Z)
• Contextual Decision Trees [62.997667081978825]
  We propose a multi-armed contextual bandit recommendation framework for feature-based selection of a single shallow tree of the learned ensemble. The trained system, which works on top of the Random Forest, dynamically identifies a base predictor that is responsible for providing the final output.
  arXiv  Detail & Related papers  (2022-07-13T17:05:08Z)
• Explaining random forest prediction through diverse rulesets [0.0]
  Local Tree eXtractor (LTreeX) is able to explain the forest prediction for a given test instance with a few diverse rules. We show that our proposed approach substantially outperforms other explainable methods in terms of predictive performance.
  arXiv  Detail & Related papers  (2022-03-29T12:54:57Z)
• Succinct Explanations With Cascading Decision Trees [5.877164140116815]
  We propose a new decision tree model that we call Cascading Decision Trees. Our key insight is to separate the notion of a decision path and an explanation path. Applying cascading decision trees to new samples results in a significantly shorter and succinct explanation.
  arXiv  Detail & Related papers  (2020-10-13T18:48:39Z)
• Rectified Decision Trees: Exploring the Landscape of Interpretable and Effective Machine Learning [66.01622034708319]
  We propose a knowledge distillation based decision trees extension, dubbed rectified decision trees (ReDT) We extend the splitting criteria and the ending condition of the standard decision trees, which allows training with soft labels. We then train the ReDT based on the soft label distilled from a well-trained teacher model through a novel jackknife-based method.
  arXiv  Detail & Related papers  (2020-08-21T10:45:25Z)
• MurTree: Optimal Classification Trees via Dynamic Programming and Search [61.817059565926336]
  We present a novel algorithm for learning optimal classification trees based on dynamic programming and search. Our approach uses only a fraction of the time required by the state-of-the-art and can handle datasets with tens of thousands of instances.
  arXiv  Detail & Related papers  (2020-07-24T17:06:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.