Unity Forests: Improving Interaction Modelling and Interpretability in Random Forests

• URL: http://arxiv.org/abs/2601.07003v1
• Date: Sun, 11 Jan 2026 17:36:58 GMT
• Title: Unity Forests: Improving Interaction Modelling and Interpretability in Random Forests
• Abstract summary: Random forests (RFs) are widely used for prediction and variable importance analysis.<n>In UFOs, the first few splits of each tree are optimized jointly.<n>UFOs achieved higher discrimination and predictive accuracy than standard RFs.
• Score: 45.88028371034407
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Random forests (RFs) are widely used for prediction and variable importance analysis and are often believed to capture any types of interactions via recursive splitting. However, since the splits are chosen locally, interactions are only reliably captured when at least one involved covariate has a marginal effect. We introduce unity forests (UFOs), an RF variant designed to better exploit interactions involving covariates without marginal effects. In UFOs, the first few splits of each tree are optimized jointly across a random covariate subset to form a "tree root" capturing such interactions; the remainder is grown conventionally. We further propose the unity variable importance measure (VIM), which is based on out-of-bag split criterion values from the tree roots. Here, only a small fraction of tree root splits with the highest in-bag criterion values are considered per covariate, reflecting that covariates with purely interaction-based effects are discriminative only if a split in an interacting covariate occurred earlier in the tree. Finally, we introduce covariate-representative tree roots (CRTRs), which select representative tree roots per covariate and provide interpretable insight into the conditions - marginal or interactive - under which each covariate has its strongest effects. In a simulation study, the unity VIM reliably identified interacting covariates without marginal effects, unlike conventional RF-based VIMs. In a large-scale real-data comparison, UFOs achieved higher discrimination and predictive accuracy than standard RFs, with comparable calibration. The CRTRs reproduced the covariates' true effect types reliably in simulated data and provided interesting insights in a real data analysis.

Related papers

• Learning Order Forest for Qualitative-Attribute Data Clustering [52.612779710298526]
  This paper discovers a tree-like distance structure to flexibly represent the local order relationship among intra-attribute qualitative values.<n>A joint learning mechanism is proposed to iteratively obtain more appropriate tree structures and clusters.<n>Experiments demonstrate that the joint learning adapts the forest to the clustering task to yield accurate results.
  arXiv  Detail & Related papers  (2026-03-03T07:49:50Z)
• Tree Ensemble Explainability through the Hoeffding Functional Decomposition and TreeHFD Algorithm [2.242085086643166]
  We introduce the TreeHFD algorithm to estimate the Hoeffding decomposition of a tree ensemble from a data sample.<n>The high performance of TreeHFD is demonstrated through experiments on both simulated and real data.
  arXiv  Detail & Related papers  (2025-10-28T09:49:01Z)
• ZTree: A Subgroup Identification Based Decision Tree Learning Framework [3.119681354260829]
  We propose ZTree, a novel decision tree learning framework.<n>It replaces CART's traditional purity based splitting with statistically principled subgroup identification.<n>ZTree consistently delivers strong performance, especially at low data regimes.
  arXiv  Detail & Related papers  (2025-09-16T05:25:16Z)
• Diversity Conscious Refined Random Forest [0.0]
  Random Forest (RF) is a widely used ensemble learning technique.<n>RF often relies on hundreds of trees and all input features, leading to high cost and model redundancy.<n>We propose a Refined Random Forest that grows trees only on informative features and enforces maximal diversity by clustering and retaining uncorrelated trees.
  arXiv  Detail & Related papers  (2025-07-01T06:28:15Z)
• Collaborative Value Function Estimation Under Model Mismatch: A Federated Temporal Difference Analysis [55.13545823385091]
  Federated reinforcement learning (FedRL) enables collaborative learning while preserving data privacy by preventing direct data exchange between agents.<n>In real-world applications, each agent may experience slightly different transition dynamics, leading to inherent model mismatches.<n>We show that even moderate levels of information sharing significantly mitigate environment-specific errors.
  arXiv  Detail & Related papers  (2025-03-21T18:06:28Z)
• Heterogeneous Random Forest [2.0646127669654835]
  Heterogeneous Random Forest (HRF) is designed to enhance tree diversity in a meaningful way. HRF consistently outperformed other ensemble methods in terms of accuracy across the majority of datasets.
  arXiv  Detail & Related papers  (2024-10-24T09:18:55Z)
• Multi forests: Variable importance for multi-class outcomes [49.1574468325115]
  We introduce a new type of variable importance measure (VIM) based on a novel RF variant called multi forests (MuFs) We show that MuFs have slightly lower predictive performance compared to conventional RFs. This is, however, not a limiting factor given the algorithm's primary purpose of calculating the multi-class VIM.
  arXiv  Detail & Related papers  (2024-09-13T15:40:29Z)
• Pure interaction effects unseen by Random Forests [0.3749861135832073]
  Random Forests are widely claimed to capture interactions well.<n>Some simple examples suggest that they perform poorly in the presence of certain pure interactions.<n>It is argued that simple alternative partitioning schemes used in the tree growing procedure can enhance identification of these interactions.
  arXiv  Detail & Related papers  (2024-06-19T12:07:22Z)
• Examining and Combating Spurious Features under Distribution Shift [94.31956965507085]
  We define and analyze robust and spurious representations using the information-theoretic concept of minimal sufficient statistics. We prove that even when there is only bias of the input distribution, models can still pick up spurious features from their training data. Inspired by our analysis, we demonstrate that group DRO can fail when groups do not directly account for various spurious correlations.
  arXiv  Detail & Related papers  (2021-06-14T05:39:09Z)
• Forest R-CNN: Large-Vocabulary Long-Tailed Object Detection and Instance Segmentation [75.93960390191262]
  We exploit prior knowledge of the relations among object categories to cluster fine-grained classes into coarser parent classes. We propose a simple yet effective resampling method, NMS Resampling, to re-balance the data distribution. Our method, termed as Forest R-CNN, can serve as a plug-and-play module being applied to most object recognition models.
  arXiv  Detail & Related papers  (2020-08-13T03:52:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.