Approximation-Free Differentiable Oblique Decision Trees

• URL: http://arxiv.org/abs/2605.07837v1
• Date: Fri, 08 May 2026 15:04:04 GMT
• Title: Approximation-Free Differentiable Oblique Decision Trees
• Abstract summary: Training accurate oblique Decision Trees (DTs) is challenging due to complex optimization landscapes and overfitting risks.<n>We propose DTSemNet, a novel, semantically equivalent, and invertible representation of hard oblique DTs as neural networks.<n>We show that DTSemNet-trained oblique DTs outperform state-of-the-art differentiable DTs.
• Score: 3.224547255825731
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Decision Trees (DTs) are widely used in safety-critical domains such as medical diagnosis, valued for their interpretability and effectiveness on tabular data. However, training accurate oblique DTs is challenging due to complex optimization landscapes and overfitting risks, particularly in regression. Recent advances have introduced differentiable formulations that enable gradient-based training and joint optimization of decision boundaries and leaf regressors. Yet, existing approaches typically rely on approximations, either through probabilistic softening of boundaries (soft DTs) or quantized gradients such as the Straight-Through Estimator (STE). To overcome these limitations, we propose DTSemNet, a novel, semantically equivalent, and invertible representation of hard oblique DTs as neural networks. DTSemNet enables end-to-end training with standard gradient descent, eliminating the need for approximations in both classification and regression. While classification aligns naturally with this formulation, regression remains challenging due to the joint optimization of internal nodes and leaf regressors. To address this, we analyze the limitations of STE and introduce an annealed Top-k method that provides accurate gradient signals without approximation. Extensive experiments on classification and regression benchmarks show that DTSemNet-trained oblique DTs outperform state-of-the-art differentiable DTs. Furthermore, we demonstrate that DTSemNet can serve as programmatic DT policies in reinforcement learning environments, thereby broadening their applicability.

Related papers

• Test-time Adaptation of Pelvic Bone Segmentation Models via Dynamic Reliability-Guided [62.10276009853225]
  Reliable pelvic bone segmentation from CT is essential for robot-assisted pelvic trauma surgery.<n>Existing methods show limited effectiveness for PBS.<n>We propose a novel closed-loop dynamic Reliability-Guided framework (ReGA) for PBS.
  arXiv  Detail & Related papers  (2026-08-01T08:13:20Z)
• Clipping Bottleneck: Stabilizing RLVR via Stochastic Recovery of Near-Boundary Signals [83.0127582612634]
  Near-boundary Rescue (NSR) is a minimal, plug-and-play modification that retains out-of-bound tokens to recover lost signals.<n>NSR substantially improves training stability and delivers consistent gains over strong baselines such as DAPO and GSPO.
  arXiv  Detail & Related papers  (2026-05-21T16:45:31Z)
• Pitfalls of Unlabeled Disagreement-Based Drift Detection in Streaming Tree Ensembles [1.2599533416395763]
  disagreement-based uncertainty has shown promise in neural networks, but its adaptation to ensembles of incremental decision trees (IDTs) remains largely unexplored.<n>We investigate this approach by constructing batch-specific disagreement measures via label flipping in ensemble members.<n>Our experiments show that, although this method performs well in ensembles of multi-layer perceptrons (MLPs), it consistently underperforms loss-based detectors when applied to IDTs.
  arXiv  Detail & Related papers  (2026-05-12T22:53:03Z)
• Decentralized Nonconvex Composite Federated Learning with Gradient Tracking and Momentum [78.27945336558987]
  Decentralized server (DFL) eliminates reliance on client-client architecture.<n>Non-smooth regularization is often incorporated into machine learning tasks.<n>We propose a novel novel DNCFL algorithm to solve these problems.
  arXiv  Detail & Related papers  (2025-04-17T08:32:25Z)
• Interpretable Deep Regression Models with Interval-Censored Failure Time Data [1.2993568435938014]
  Deep learning methods for interval-censored data remain underexplored and limited to specific data type or model.<n>This work proposes a general regression framework for interval-censored data with a broad class of partially linear transformation models.<n>Applying our method to the Alzheimer's Disease Neuroimaging Initiative dataset yields novel insights and improved predictive performance compared to traditional approaches.
  arXiv  Detail & Related papers  (2025-03-25T15:27:32Z)
• Dual Cone Gradient Descent for Training Physics-Informed Neural Networks [0.0]
  Physics-informed dual neural networks (PINNs) have emerged as a prominent approach for solving partial differential equations.<n>We propose a novel framework, Dual Cone Gradient Descent (DCGD), which adjusts the direction of the updated gradient to ensure it falls within a cone region.
  arXiv  Detail & Related papers  (2024-09-27T03:27:46Z)
• Complexity-Aware Deep Symbolic Regression with Robust Risk-Seeking Policy Gradients [20.941908494137806]
  We propose a novel deep symbolic regression approach to enhance the robustness and interpretability of data-driven mathematical expression discovery.<n>Our work is aligned with the popular DSR framework which focuses on learning a data-specific expression generator.
  arXiv  Detail & Related papers  (2024-06-10T19:29:10Z)
• Understanding the robustness difference between stochastic gradient descent and adaptive gradient methods [11.895321856533934]
  gradient descent (SGD) and adaptive gradient methods have been widely used in training deep neural networks. We empirically show that while the difference between the standard generalization performance of models trained using these methods is small, those trained using SGD exhibit far greater robustness under input perturbations.
  arXiv  Detail & Related papers  (2023-08-13T07:03:22Z)
• Non-adversarial training of Neural SDEs with signature kernel scores [4.721845865189578]
  State-of-the-art performance for irregular time series generation has been previously obtained by training these models adversarially as GANs. In this paper, we introduce a novel class of scoring rules on pathspace based on signature kernels.
  arXiv  Detail & Related papers  (2023-05-25T17:31:18Z)
• Implicit Stochastic Gradient Descent for Training Physics-informed Neural Networks [51.92362217307946]
  Physics-informed neural networks (PINNs) have effectively been demonstrated in solving forward and inverse differential equation problems. PINNs are trapped in training failures when the target functions to be approximated exhibit high-frequency or multi-scale features. In this paper, we propose to employ implicit gradient descent (ISGD) method to train PINNs for improving the stability of training process.
  arXiv  Detail & Related papers  (2023-03-03T08:17:47Z)
• Truncated tensor Schatten p-norm based approach for spatiotemporal traffic data imputation with complicated missing patterns [77.34726150561087]
  We introduce four complicated missing patterns, including missing and three fiber-like missing cases according to the mode-drivenn fibers. Despite nonity of the objective function in our model, we derive the optimal solutions by integrating alternating data-mputation method of multipliers.
  arXiv  Detail & Related papers  (2022-05-19T08:37:56Z)
• Improving Generalization via Uncertainty Driven Perturbations [107.45752065285821]
  We consider uncertainty-driven perturbations of the training data points. Unlike loss-driven perturbations, uncertainty-guided perturbations do not cross the decision boundary. We show that UDP is guaranteed to achieve the robustness margin decision on linear models.
  arXiv  Detail & Related papers  (2022-02-11T16:22:08Z)
• Adversarial Training for Gradient Descent: Analysis Through its Continuous-time Approximation [11.389868267342491]
  Adversarial training has gained great popularity as one of the most effective defenses for deep neural network. This paper establishes a continuous-time approximation for the mini-max game of adversarial training.
  arXiv  Detail & Related papers  (2021-05-17T17:41:56Z)
• AdaS: Adaptive Scheduling of Stochastic Gradients [50.80697760166045]
  We introduce the notions of textit"knowledge gain" and textit"mapping condition" and propose a new algorithm called Adaptive Scheduling (AdaS) Experimentation reveals that, using the derived metrics, AdaS exhibits: (a) faster convergence and superior generalization over existing adaptive learning methods; and (b) lack of dependence on a validation set to determine when to stop training.
  arXiv  Detail & Related papers  (2020-06-11T16:36:31Z)
• Extrapolation for Large-batch Training in Deep Learning [72.61259487233214]
  We show that a host of variations can be covered in a unified framework that we propose. We prove the convergence of this novel scheme and rigorously evaluate its empirical performance on ResNet, LSTM, and Transformer.
  arXiv  Detail & Related papers  (2020-06-10T08:22:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.