Treeffuser: Probabilistic Predictions via Conditional Diffusions with Gradient-Boosted Trees

• URL: http://arxiv.org/abs/2406.07658v2
• Date: Tue, 22 Oct 2024 04:29:19 GMT
• Title: Treeffuser: Probabilistic Predictions via Conditional Diffusions with Gradient-Boosted Trees
• Abstract summary: Treeffuser is an easy-to-use method for probabilistic prediction on tabular data. Treeffuser learns well-calibrated predictive distributions and can handle a wide range of regression tasks. We demonstrate its versatility with an application to inventory allocation under uncertainty using sales data from Walmart.
• Score: 39.9546129327526
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Probabilistic prediction aims to compute predictive distributions rather than single point predictions. These distributions enable practitioners to quantify uncertainty, compute risk, and detect outliers. However, most probabilistic methods assume parametric responses, such as Gaussian or Poisson distributions. When these assumptions fail, such models lead to bad predictions and poorly calibrated uncertainty. In this paper, we propose Treeffuser, an easy-to-use method for probabilistic prediction on tabular data. The idea is to learn a conditional diffusion model where the score function is estimated using gradient-boosted trees. The conditional diffusion model makes Treeffuser flexible and non-parametric, while the gradient-boosted trees make it robust and easy to train on CPUs. Treeffuser learns well-calibrated predictive distributions and can handle a wide range of regression tasks -- including those with multivariate, multimodal, and skewed responses. We study Treeffuser on synthetic and real data and show that it outperforms existing methods, providing better calibrated probabilistic predictions. We further demonstrate its versatility with an application to inventory allocation under uncertainty using sales data from Walmart. We implement Treeffuser in https://github.com/blei-lab/treeffuser.

Related papers

• Sample-Adaptive Latent Rewards for Uncertainty-Guided Diffusion Post-Training [51.18425726646089]
  We propose textscSURE, a unified latent-space framework for image and video diffusion models.<n>It learns reward distributions and directly uses their reliability to guide dense post-training.<n>textscSURE-REFL achieves the highest VBench quality, semantic, and total scores among the evaluated methods.
  arXiv  Detail & Related papers  (2026-08-06T14:55:42Z)
• Rethinking Likelihood distributions: Student's t Likelihood Boosts Bayesian Neural Network Performance [0.0]
  We investigate whether alternative assumptions for the likelihood distribution can outperform the commonly used Gaussian.<n>Student's t yields better predictive performance than a Gaussian likelihood distribution, independent of the data distribution and architecture.<n>In some cases, Student's t can also lead to shorter training times, while still being easy to implement.
  arXiv  Detail & Related papers  (2026-07-28T07:32:02Z)
• d-TreeRPO: Towards More Reliable Policy Optimization for Diffusion Language Models [45.27333046908981]
  emphd-TreeRPO is a reliable reinforcement learning framework for dLLMs.<n>We show that emphd-TreeRPO achieves significant gains on multiple reasoning benchmarks.
  arXiv  Detail & Related papers  (2025-12-10T14:20:07Z)
• Efficient distributional regression trees learning algorithms for calibrated non-parametric probabilistic forecasts [0.0]
  In the context of regression, instead of estimating a conditional mean, this can be achieved by producing a predictive interval for the output. This paper introduces novel algorithms for learning probabilistic regression trees for the WIS or CRPS loss functions.
  arXiv  Detail & Related papers  (2025-02-07T18:39:35Z)
• Rejection via Learning Density Ratios [50.91522897152437]
  Classification with rejection emerges as a learning paradigm which allows models to abstain from making predictions. We propose a different distributional perspective, where we seek to find an idealized data distribution which maximizes a pretrained model's performance. Our framework is tested empirically over clean and noisy datasets.
  arXiv  Detail & Related papers  (2024-05-29T01:32:17Z)
• Regression Trees for Fast and Adaptive Prediction Intervals [2.6763498831034043]
  We present a family of methods to calibrate prediction intervals for regression problems with local coverage guarantees. We create a partition by training regression trees and Random Forests on conformity scores. Our proposal is versatile, as it applies to various conformity scores and prediction settings.
  arXiv  Detail & Related papers  (2024-02-12T01:17:09Z)
• User-defined Event Sampling and Uncertainty Quantification in Diffusion Models for Physical Dynamical Systems [49.75149094527068]
  We show that diffusion models can be adapted to make predictions and provide uncertainty quantification for chaotic dynamical systems. We develop a probabilistic approximation scheme for the conditional score function which converges to the true distribution as the noise level decreases. We are able to sample conditionally on nonlinear userdefined events at inference time, and matches data statistics even when sampling from the tails of the distribution.
  arXiv  Detail & Related papers  (2023-06-13T03:42:03Z)
• Performative Prediction with Neural Networks [24.880495520422]
  performative prediction is a framework for learning models that influence the data they intend to predict. Standard convergence results for finding a performatively stable classifier with the method of repeated risk minimization assume that the data distribution is Lipschitz continuous to the model's parameters. In this work, we instead assume that the data distribution is Lipschitz continuous with respect to the model's predictions, a more natural assumption for performative systems.
  arXiv  Detail & Related papers  (2023-04-14T01:12:48Z)
• Distributional Gradient Boosting Machines [77.34726150561087]
  Our framework is based on XGBoost and LightGBM. We show that our framework achieves state-of-the-art forecast accuracy.
  arXiv  Detail & Related papers  (2022-04-02T06:32:19Z)
• Multivariate Probabilistic Regression with Natural Gradient Boosting [63.58097881421937]
  We propose a Natural Gradient Boosting (NGBoost) approach based on nonparametrically modeling the conditional parameters of the multivariate predictive distribution. Our method is robust, works out-of-the-box without extensive tuning, is modular with respect to the assumed target distribution, and performs competitively in comparison to existing approaches.
  arXiv  Detail & Related papers  (2021-06-07T17:44:49Z)
• Probabilistic Gradient Boosting Machines for Large-Scale Probabilistic Regression [51.770998056563094]
  Probabilistic Gradient Boosting Machines (PGBM) is a method to create probabilistic predictions with a single ensemble of decision trees. We empirically demonstrate the advantages of PGBM compared to existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-06-03T08:32:13Z)
• Handling Missing Data in Decision Trees: A Probabilistic Approach [41.259097100704324]
  We tackle the problem of handling missing data in decision trees by taking a probabilistic approach. We use tractable density estimators to compute the "expected prediction" of our models. At learning time, we fine-tune parameters of already learned trees by minimizing their "expected prediction loss"
  arXiv  Detail & Related papers  (2020-06-29T19:54:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.