Verifying Tree Ensembles by Reasoning about Potential Instances
- URL: http://arxiv.org/abs/2001.11905v3
- Date: Tue, 18 May 2021 12:54:32 GMT
- Title: Verifying Tree Ensembles by Reasoning about Potential Instances
- Authors: Laurens Devos, Wannes Meert, Jesse Davis
- Abstract summary: We present a strategy that can prune part of the input space given the question asked to simplify the problem.
We then follow a divide and conquer approach that is incremental and can always return some answers.
The usefulness of our approach is shown on a diverse set of use cases.
- Score: 25.204157642042627
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Imagine being able to ask questions to a black box model such as "Which
adversarial examples exist?", "Does a specific attribute have a
disproportionate effect on the model's prediction?" or "What kind of
predictions could possibly be made for a partially described example?" This
last question is particularly important if your partial description does not
correspond to any observed example in your data, as it provides insight into
how the model will extrapolate to unseen data. These capabilities would be
extremely helpful as they would allow a user to better understand the model's
behavior, particularly as it relates to issues such as robustness, fairness,
and bias. In this paper, we propose such an approach for an ensemble of trees.
Since, in general, this task is intractable we present a strategy that (1) can
prune part of the input space given the question asked to simplify the problem;
and (2) follows a divide and conquer approach that is incremental and can
always return some answers and indicates which parts of the input domains are
still uncertain. The usefulness of our approach is shown on a diverse set of
use cases.
Related papers
- What if you said that differently?: How Explanation Formats Affect Human Feedback Efficacy and User Perception [53.4840989321394]
We analyze the effect of rationales generated by QA models to support their answers.
We present users with incorrect answers and corresponding rationales in various formats.
We measure the effectiveness of this feedback in patching these rationales through in-context learning.
arXiv Detail & Related papers (2023-11-16T04:26:32Z) - Understanding Text Classification Data and Models Using Aggregated Input
Salience [2.105564340986074]
In some cases, an input salience method, which highlights the most important parts of the input, may reveal problematic reasoning.
In this paper we aim to address these issues and go from understanding single examples to understanding entire datasets and models.
Using this methodology we address multiple distinct but common model developer needs by showing how problematic data and model behavior can be identified.
arXiv Detail & Related papers (2022-11-10T11:00:57Z) - NaturalAdversaries: Can Naturalistic Adversaries Be as Effective as
Artificial Adversaries? [61.58261351116679]
We introduce a two-stage adversarial example generation framework (NaturalAdversaries) for natural language understanding tasks.
It is adaptable to both black-box and white-box adversarial attacks based on the level of access to the model parameters.
Our results indicate these adversaries generalize across domains, and offer insights for future research on improving robustness of neural text classification models.
arXiv Detail & Related papers (2022-11-08T16:37:34Z) - Can I see an Example? Active Learning the Long Tail of Attributes and
Relations [64.50739983632006]
We introduce a novel incremental active learning framework that asks for attributes and relations in visual scenes.
While conventional active learning methods ask for labels of specific examples, we flip this framing to allow agents to ask for examples from specific categories.
Using this framing, we introduce an active sampling method that asks for examples from the tail of the data distribution and show that it outperforms classical active learning methods on Visual Genome.
arXiv Detail & Related papers (2022-03-11T19:28:19Z) - Contrastive Explanations for Model Interpretability [77.92370750072831]
We propose a methodology to produce contrastive explanations for classification models.
Our method is based on projecting model representation to a latent space.
Our findings shed light on the ability of label-contrastive explanations to provide a more accurate and finer-grained interpretability of a model's decision.
arXiv Detail & Related papers (2021-03-02T00:36:45Z) - Counterfactual Explanations for Oblique Decision Trees: Exact, Efficient
Algorithms [0.0]
We consider counterfactual explanations, the problem of minimally adjusting features in a source input instance so that it is classified as a target class under a given classification.
This has become a topic of recent interest as a way to query a trained model and suggest possible actions to overturn its decision.
arXiv Detail & Related papers (2021-03-01T16:04:33Z) - Dependency Decomposition and a Reject Option for Explainable Models [4.94950858749529]
Recent deep learning models perform extremely well in various inference tasks.
Recent advances offer methods to visualize features, describe attribution of the input.
We present the first analysis of dependencies regarding the probability distribution over the desired image classification outputs.
arXiv Detail & Related papers (2020-12-11T17:39:33Z) - Towards Unifying Feature Attribution and Counterfactual Explanations:
Different Means to the Same End [17.226134854746267]
We present a method to generate feature attribution explanations from a set of counterfactual examples.
We show how counterfactual examples can be used to evaluate the goodness of an attribution-based explanation in terms of its necessity and sufficiency.
arXiv Detail & Related papers (2020-11-10T05:41:43Z) - Deducing neighborhoods of classes from a fitted model [68.8204255655161]
In this article a new kind of interpretable machine learning method is presented.
It can help to understand the partitioning of the feature space into predicted classes in a classification model using quantile shifts.
Basically, real data points (or specific points of interest) are used and the changes of the prediction after slightly raising or decreasing specific features are observed.
arXiv Detail & Related papers (2020-09-11T16:35:53Z) - Value-driven Hindsight Modelling [68.658900923595]
Value estimation is a critical component of the reinforcement learning (RL) paradigm.
Model learning can make use of the rich transition structure present in sequences of observations, but this approach is usually not sensitive to the reward function.
We develop an approach for representation learning in RL that sits in between these two extremes.
This provides tractable prediction targets that are directly relevant for a task, and can thus accelerate learning the value function.
arXiv Detail & Related papers (2020-02-19T18:10:20Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.