Towards Robust Classification with Deep Generative Forests

• URL: http://arxiv.org/abs/2007.05721v1
• Date: Sat, 11 Jul 2020 08:57:52 GMT
• Title: Towards Robust Classification with Deep Generative Forests
• Authors: Alvaro H. C. Correia, Robert Peharz, Cassio de Campos
• Abstract summary: Decision Trees and Random Forests are among the most widely used machine learning models. Being primarily discriminative models they lack principled methods to manipulate the uncertainty of predictions. We exploit Generative Forests (GeFs) to extend Random Forests to generative models representing the full joint distribution over the feature space.
• Score: 13.096855747795303
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Decision Trees and Random Forests are among the most widely used machine learning models, and often achieve state-of-the-art performance in tabular, domain-agnostic datasets. Nonetheless, being primarily discriminative models they lack principled methods to manipulate the uncertainty of predictions. In this paper, we exploit Generative Forests (GeFs), a recent class of deep probabilistic models that addresses these issues by extending Random Forests to generative models representing the full joint distribution over the feature space. We demonstrate that GeFs are uncertainty-aware classifiers, capable of measuring the robustness of each prediction as well as detecting out-of-distribution samples.

Related papers

• Towards Generalizable and Interpretable Motion Prediction: A Deep Variational Bayes Approach [54.429396802848224]
  This paper proposes an interpretable generative model for motion prediction with robust generalizability to out-of-distribution cases. For interpretability, the model achieves the target-driven motion prediction by estimating the spatial distribution of long-term destinations. Experiments on motion prediction datasets validate that the fitted model can be interpretable and generalizable.
  arXiv  Detail & Related papers  (2024-03-10T04:16:04Z)
• Example-based Explanations for Random Forests using Machine Unlearning [4.006745047019997]
  Tree-based machine learning models, such as decision trees and random forests, have been hugely successful in classification tasks. Despite their popularity and power, these models have been found to produce unexpected or discriminatory outcomes. We introduce Fair Debugger, a system to identify training data subsets responsible for instances of fairness violations in the outcomes of a random forest classifier.
  arXiv  Detail & Related papers  (2024-02-07T16:28:04Z)
• Function-Space Regularization for Deep Bayesian Classification [33.63495888167032]
  We apply a Dirichlet prior in predictive space and perform approximate function-space variational inference. By adapting the inference, the same function-space prior can be combined with different models without affecting model architecture or size.
  arXiv  Detail & Related papers  (2023-07-12T10:17:54Z)
• Variational Classification [51.2541371924591]
  We derive a variational objective to train the model, analogous to the evidence lower bound (ELBO) used to train variational auto-encoders. Treating inputs to the softmax layer as samples of a latent variable, our abstracted perspective reveals a potential inconsistency. We induce a chosen latent distribution, instead of the implicit assumption found in a standard softmax layer.
  arXiv  Detail & Related papers  (2023-05-17T17:47:19Z)
• FACT: High-Dimensional Random Forests Inference [4.941630596191806]
  Quantifying the usefulness of individual features in random forests learning can greatly enhance its interpretability. Existing studies have shown that some popularly used feature importance measures for random forests suffer from the bias issue. We propose a framework of the self-normalized feature-residual correlation test (FACT) for evaluating the significance of a given feature.
  arXiv  Detail & Related papers  (2022-07-04T19:05:08Z)
• On Uncertainty Estimation by Tree-based Surrogate Models in Sequential Model-based Optimization [13.52611859628841]
  We revisit various ensembles of randomized trees to investigate their behavior in the perspective of prediction uncertainty estimation. We propose a new way of constructing an ensemble of randomized trees, referred to as BwO forest, where bagging with oversampling is employed to construct bootstrapped samples. Experimental results demonstrate the validity and good performance of BwO forest over existing tree-based models in various circumstances.
  arXiv  Detail & Related papers  (2022-02-22T04:50:37Z)
• NUQ: Nonparametric Uncertainty Quantification for Deterministic Neural Networks [151.03112356092575]
  We show the principled way to measure the uncertainty of predictions for a classifier based on Nadaraya-Watson's nonparametric estimate of the conditional label distribution. We demonstrate the strong performance of the method in uncertainty estimation tasks on a variety of real-world image datasets.
  arXiv  Detail & Related papers  (2022-02-07T12:30:45Z)
• CC-Cert: A Probabilistic Approach to Certify General Robustness of Neural Networks [58.29502185344086]
  In safety-critical machine learning applications, it is crucial to defend models against adversarial attacks. It is important to provide provable guarantees for deep learning models against semantically meaningful input transformations. We propose a new universal probabilistic certification approach based on Chernoff-Cramer bounds.
  arXiv  Detail & Related papers  (2021-09-22T12:46:04Z)
• Open Set Recognition with Conditional Probabilistic Generative Models [51.40872765917125]
  We propose Conditional Probabilistic Generative Models (CPGM) for open set recognition. CPGM can detect unknown samples but also classify known classes by forcing different latent features to approximate conditional Gaussian distributions. Experiment results on multiple benchmark datasets reveal that the proposed method significantly outperforms the baselines.
  arXiv  Detail & Related papers  (2020-08-12T06:23:49Z)
• Unlabelled Data Improves Bayesian Uncertainty Calibration under Covariate Shift [100.52588638477862]
  We develop an approximate Bayesian inference scheme based on posterior regularisation. We demonstrate the utility of our method in the context of transferring prognostic models of prostate cancer across globally diverse populations.
  arXiv  Detail & Related papers  (2020-06-26T13:50:19Z)
• Joints in Random Forests [13.096855747795303]
  Decision Trees (DTs) and Random Forests (RFs) are powerful discriminative learners and tools of central importance to the everyday machine learning practitioner and data scientist. We show that DTs and RFs can naturally be interpreted as generative models, by drawing a connection to Probabilistic Circuits. This reinterpretation equips them with a full joint distribution over the feature space and leads to Generative Decision Trees (GeDTs) and Generative Forests (GeFs)
  arXiv  Detail & Related papers  (2020-06-25T14:17:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.