Interpretable Differencing of Machine Learning Models
- URL: http://arxiv.org/abs/2306.06473v2
- Date: Tue, 13 Jun 2023 05:23:19 GMT
- Title: Interpretable Differencing of Machine Learning Models
- Authors: Swagatam Haldar, Diptikalyan Saha, Dennis Wei, Rahul Nair, Elizabeth
M. Daly
- Abstract summary: We formalize the problem of model differencing as one of predicting a dissimilarity function of two ML models' outputs.
A Joint Surrogate Tree (JST) is composed of two conjoined decision tree surrogates for the two models.
A JST provides an intuitive representation of differences and places the changes in the context of the models' decision logic.
- Score: 20.99877540751412
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Understanding the differences between machine learning (ML) models is of
interest in scenarios ranging from choosing amongst a set of competing models,
to updating a deployed model with new training data. In these cases, we wish to
go beyond differences in overall metrics such as accuracy to identify where in
the feature space do the differences occur. We formalize this problem of model
differencing as one of predicting a dissimilarity function of two ML models'
outputs, subject to the representation of the differences being
human-interpretable. Our solution is to learn a Joint Surrogate Tree (JST),
which is composed of two conjoined decision tree surrogates for the two models.
A JST provides an intuitive representation of differences and places the
changes in the context of the models' decision logic. Context is important as
it helps users to map differences to an underlying mental model of an AI
system. We also propose a refinement procedure to increase the precision of a
JST. We demonstrate, through an empirical evaluation, that such contextual
differencing is concise and can be achieved with no loss in fidelity over naive
approaches.
Related papers
- Revisiting Demonstration Selection Strategies in In-Context Learning [66.11652803887284]
Large language models (LLMs) have shown an impressive ability to perform a wide range of tasks using in-context learning (ICL)
In this work, we first revisit the factors contributing to this variance from both data and model aspects, and find that the choice of demonstration is both data- and model-dependent.
We propose a data- and model-dependent demonstration selection method, textbfTopK + ConE, based on the assumption that textitthe performance of a demonstration positively correlates with its contribution to the model's understanding of the test samples.
arXiv Detail & Related papers (2024-01-22T16:25:27Z) - Dynamic Interpretability for Model Comparison via Decision Rules [0.8099700053397276]
This paper addresses the challenge of understanding and explaining differences between machine learning models.
We propose DeltaXplainer, a model-agnostic method for generating rule-based explanations.
arXiv Detail & Related papers (2023-09-29T09:42:49Z) - ModelDiff: A Framework for Comparing Learning Algorithms [86.19580801269036]
We study the problem of (learning) algorithm comparison, where the goal is to find differences between models trained with two different learning algorithms.
We present ModelDiff, a method that leverages the datamodels framework to compare learning algorithms based on how they use their training data.
arXiv Detail & Related papers (2022-11-22T18:56:52Z) - Statistical Aspects of SHAP: Functional ANOVA for Model Interpretation [0.456877715768796]
We show that challenges in SHAP approximations largely relate to the choice of a feature distribution and the number of $2p$ ANOVA terms estimated.
We argue that the connection between machine learning explainability and sensitivity analysis is illuminating in this case, but the immediate practical consequences are not obvious.
arXiv Detail & Related papers (2022-08-21T21:46:15Z) - Switchable Representation Learning Framework with Self-compatibility [50.48336074436792]
We propose a Switchable representation learning Framework with Self-Compatibility (SFSC)
SFSC generates a series of compatible sub-models with different capacities through one training process.
SFSC achieves state-of-the-art performance on the evaluated datasets.
arXiv Detail & Related papers (2022-06-16T16:46:32Z) - IMACS: Image Model Attribution Comparison Summaries [16.80986701058596]
We introduce IMACS, a method that combines gradient-based model attributions with aggregation and visualization techniques.
IMACS extracts salient input features from an evaluation dataset, clusters them based on similarity, then visualizes differences in model attributions for similar input features.
We show how our technique can uncover behavioral differences caused by domain shift between two models trained on satellite images.
arXiv Detail & Related papers (2022-01-26T21:35:14Z) - Multivariate Data Explanation by Jumping Emerging Patterns Visualization [78.6363825307044]
We present VAX (multiVariate dAta eXplanation), a new VA method to support the identification and visual interpretation of patterns in multivariate data sets.
Unlike the existing similar approaches, VAX uses the concept of Jumping Emerging Patterns to identify and aggregate several diversified patterns, producing explanations through logic combinations of data variables.
arXiv Detail & Related papers (2021-06-21T13:49:44Z) - Beyond Trivial Counterfactual Explanations with Diverse Valuable
Explanations [64.85696493596821]
In computer vision applications, generative counterfactual methods indicate how to perturb a model's input to change its prediction.
We propose a counterfactual method that learns a perturbation in a disentangled latent space that is constrained using a diversity-enforcing loss.
Our model improves the success rate of producing high-quality valuable explanations when compared to previous state-of-the-art methods.
arXiv Detail & Related papers (2021-03-18T12:57:34Z) - Interpretable Multi-dataset Evaluation for Named Entity Recognition [110.64368106131062]
We present a general methodology for interpretable evaluation for the named entity recognition (NER) task.
The proposed evaluation method enables us to interpret the differences in models and datasets, as well as the interplay between them.
By making our analysis tool available, we make it easy for future researchers to run similar analyses and drive progress in this area.
arXiv Detail & Related papers (2020-11-13T10:53:27Z) - Learning Invariances for Interpretability using Supervised VAE [0.0]
We learn model invariances as a means of interpreting a model.
We propose a supervised form of variational auto-encoders (VAEs)
We show how combining our model with feature attribution methods it is possible to reach a more fine-grained understanding about the decision process of the model.
arXiv Detail & Related papers (2020-07-15T10:14:16Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.