Twoblock clustering trees with coskewness-based dimension reduction: recovering piecewise multivariate linear regimes

• URL: http://arxiv.org/abs/2607.20760v1
• Date: Wed, 22 Jul 2026 22:20:42 GMT
• Title: Twoblock clustering trees with coskewness-based dimension reduction: recovering piecewise multivariate linear regimes
• Abstract summary: The twoblock clustering tree (tbtree) is introduced as a highly interpretable regression tree for multivariate responses.<n>Twoblock trees are deterministic decision trees that have local multivariate linear models as their leaves.<n>The twoblock models that generate the splits, as well as the ones in the leaves, can be inspected and interpreted.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The twoblock clustering tree (\tbtree) is introduced as a highly interpretable regression tree for multivariate responses. Twoblock trees are deterministic decision trees that have local multivariate linear models as their leaves and use dense or sparse twoblock dimension reduction as local leaf models and in the impurity. The resulting models are both computationally efficient and can be highly interpretable. Beyond proposing the decision tree estimator itself, this paper also introduces an estimator for the twoblock dimension reduced space based on maximizing coskewness, which facilitates identification of non-normal clusters in the data. The tree inherently produces a set of local linear models and is therefore apt to recover peicewise linear regimes, which is illustrated in a simulation. However, two real world data examples illustrate that twoblock trees are also capable of modeling more complexly nonlinear dependencies and can perform on par with black box modeling techniques, such as random forests. At each point, both the twoblock models that generate the splits, as well as the ones in the leaves, can be inspected and interpreted.

Related papers

• Partition Trees: Conditional Density Estimation over General Outcome Spaces [46.1988967916659]
  We propose Partition Trees, a tree-based framework for conditional density estimation over general outcome spaces.<n>Our approach models conditional distributions as piecewise-constant densities on data adaptive partitions and learns trees by directly minimizing conditional negative log-likelihood.
  arXiv  Detail & Related papers  (2026-02-03T22:12:30Z)
• Hierarchical Quantized Diffusion Based Tree Generation Method for Hierarchical Representation and Lineage Analysis [49.00783841494125]
  HDTree captures tree relationships within a hierarchical latent space using a unified hierarchical codebook and quantized diffusion processes.<n> HDTree's effectiveness is demonstrated through comparisons on both general-purpose and single-cell datasets.<n>These contributions provide a new tool for hierarchical lineage analysis, enabling more accurate and efficient modeling of cellular differentiation paths.
  arXiv  Detail & Related papers  (2025-06-29T15:19:13Z)
• Experiments with Optimal Model Trees [1.3750624267664155]
  We show that globally optimal model trees can achieve competitive accuracy with very small trees.<n>We also compare to classic optimal and greedily grown decision trees, random forests, and support vector machines.
  arXiv  Detail & Related papers  (2025-03-17T08:03:47Z)
• Learning Decision Trees as Amortized Structure Inference [59.65621207449269]
  We propose a hybrid amortized structure inference approach to learn predictive decision tree ensembles given data.<n>We show that our approach, DT-GFN, outperforms state-of-the-art decision tree and deep learning methods on standard classification benchmarks.
  arXiv  Detail & Related papers  (2025-03-10T07:05:07Z)
• Explainable Clustering Beyond Worst-Case Guarantees [5.65604054654671]
  We study the explainable clustering problem first posed by Moshkovitz, Dasgupta, Rashtchian, and Frost (ICML 2020)<n>The goal of explainable clustering is to fit an axis-aligned decision tree with $K$ leaves and minimal clustering cost (where every leaf is a cluster)
  arXiv  Detail & Related papers  (2024-11-03T14:00:20Z)
• Learning on Model Weights using Tree Experts [39.90685550999956]
  Training machine learning models to infer missing documentation directly from model weights is challenging.<n>We identify a key property of real-world models: most public models belong to a small set of Model Trees.<n>We introduce Probing Experts (ProbeX), a theoretically motivated and lightweight method to learn from the weights of a single model layer.
  arXiv  Detail & Related papers  (2024-10-17T17:17:09Z)
• Exploring space efficiency in a tree-based linear model for extreme multi-label classification [11.18858602369985]
  Extreme multi-label classification (XMC) aims to identify relevant subsets from numerous labels. Among the various approaches for XMC, tree-based linear models are effective due to their superior efficiency and simplicity. In this work, we conduct both theoretical and empirical analyses on the space to store a tree model under the assumption of sparse data.
  arXiv  Detail & Related papers  (2024-10-12T15:02:40Z)
• Terminating Differentiable Tree Experts [77.2443883991608]
  We propose a neuro-symbolic Differentiable Tree Machine that learns tree operations using a combination of transformers and Representation Products. We first remove a series of different transformer layers that are used in every step by introducing a mixture of experts. We additionally propose a new termination algorithm to provide the model the power to choose how many steps to make automatically.
  arXiv  Detail & Related papers  (2024-07-02T08:45:38Z)
• Mixtures of All Trees [28.972995038976745]
  We propose a novel class of generative models called mixtures of all trees: that is, a mixture over all possible ($nn-2$) tree-shaped graphical models over $n$ variables. We show that it is possible to parameterize this Mixture of All Trees (MoAT) model compactly in a way that allows for tractable likelihood and optimization via gradient descent.
  arXiv  Detail & Related papers  (2023-02-27T23:37:03Z)
• On marginal feature attributions of tree-based models [0.11184789007828977]
  Local feature attributions based on marginal expectations, e.g. marginal Shapley, Owen or Banzhaf values, may be employed. We present two (statistically similar) decision trees that compute the exact same function for which the "path-dependent" TreeSHAP yields different rankings of features. We exploit the symmetry to derive an explicit formula, with improved complexity and only in terms of the internal model parameters, for marginal Shapley (and Banzhaf and Owen) values of CatBoost models.
  arXiv  Detail & Related papers  (2023-02-16T17:18:03Z)
• Robust estimation of tree structured models [0.0]
  We show that it is possible to recover trees from noisy binary data up to a small equivalence class of possible trees. We also provide a characterisation of when the Chow-Liu algorithm consistently learns the underlying tree from the noisy data.
  arXiv  Detail & Related papers  (2021-02-10T14:58:40Z)
• Provable Model-based Nonlinear Bandit and Reinforcement Learning: Shelve Optimism, Embrace Virtual Curvature [61.22680308681648]
  We show that global convergence is statistically intractable even for one-layer neural net bandit with a deterministic reward. For both nonlinear bandit and RL, the paper presents a model-based algorithm, Virtual Ascent with Online Model Learner (ViOL)
  arXiv  Detail & Related papers  (2021-02-08T12:41:56Z)
• Growing Deep Forests Efficiently with Soft Routing and Learned Connectivity [79.83903179393164]
  This paper further extends the deep forest idea in several important aspects. We employ a probabilistic tree whose nodes make probabilistic routing decisions, a.k.a., soft routing, rather than hard binary decisions. Experiments on the MNIST dataset demonstrate that our empowered deep forests can achieve better or comparable performance than [1],[3].
  arXiv  Detail & Related papers  (2020-12-29T18:05:05Z)
• Convex Polytope Trees [57.56078843831244]
  convex polytope trees (CPT) are proposed to expand the family of decision trees by an interpretable generalization of their decision boundary. We develop a greedy method to efficiently construct CPT and scalable end-to-end training algorithms for the tree parameters when the tree structure is given.
  arXiv  Detail & Related papers  (2020-10-21T19:38:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.