Synthesizing Pareto-Optimal Interpretations for Black-Box Models
- URL: http://arxiv.org/abs/2108.07307v1
- Date: Mon, 16 Aug 2021 18:33:16 GMT
- Title: Synthesizing Pareto-Optimal Interpretations for Black-Box Models
- Abstract summary: We present a new multi-objective optimization approach for synthesizing interpretations.
We show that the underlying multi-objective optimization problem can be solved via a reduction to quantitative constraint solving.
Our experiments show that there often exists a rich and varied set of choices for interpretations that are missed by existing approaches.
- Score: 7.345977965095031
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We present a new multi-objective optimization approach for synthesizing
interpretations that "explain" the behavior of black-box machine learning
models. Constructing human-understandable interpretations for black-box models
often requires balancing conflicting objectives. A simple interpretation may be
easier to understand for humans while being less precise in its predictions
vis-a-vis a complex interpretation. Existing methods for synthesizing
interpretations use a single objective function and are often optimized for a
single class of interpretations. In contrast, we provide a more general and
multi-objective synthesis framework that allows users to choose (1) the class
of syntactic templates from which an interpretation should be synthesized, and
(2) quantitative measures on both the correctness and explainability of an
interpretation. For a given black-box, our approach yields a set of
Pareto-optimal interpretations with respect to the correctness and
explainability measures. We show that the underlying multi-objective
optimization problem can be solved via a reduction to quantitative constraint
solving, such as weighted maximum satisfiability. To demonstrate the benefits
of our approach, we have applied it to synthesize interpretations for black-box
neural-network classifiers. Our experiments show that there often exists a rich
and varied set of choices for interpretations that are missed by existing
approaches.
Related papers
- Unpacking Interpretability: Human-Centered Criteria for Optimal Combinatorial Solutions [0.4077787659104315]
We present an experimental paradigm in which participants chose which of two equally optimal solutions for packing items into bins was easier to understand.<n>We show that preferences reliably track three properties of solution structure.<n>These results provide a concrete, feature-based account of interpretability in optimal packing solutions.
arXiv Detail & Related papers (2026-03-09T19:18:52Z) - Locally Pareto-Optimal Interpretations for Black-Box Machine Learning Models [0.8906214436849201]
We develop a framework based on local optimality guarantees to enable scalable synthesis of interpretations.<n>We show that our approach yields interpretations that closely match those synthesized by methods offering global guarantees.
arXiv Detail & Related papers (2025-08-21T04:11:20Z) - Beyond Interpretability: The Gains of Feature Monosemanticity on Model Robustness [68.69369585600698]
Deep learning models often suffer from a lack of interpretability due to polysemanticity.
Recent advances in monosemanticity, where neurons correspond to consistent and distinct semantics, have significantly improved interpretability.
We show that monosemantic features not only enhance interpretability but also bring concrete gains in model performance.
arXiv Detail & Related papers (2024-10-27T18:03:20Z) - InterpretCC: Intrinsic User-Centric Interpretability through Global Mixture of Experts [31.738009841932374]
Interpretability for neural networks is a trade-off between three key requirements.
We present InterpretCC, a family of interpretable-by-design neural networks that guarantee human-centric interpretability.
arXiv Detail & Related papers (2024-02-05T11:55:50Z) - Explanation-aware Soft Ensemble Empowers Large Language Model In-context
Learning [50.00090601424348]
Large language models (LLMs) have shown remarkable capabilities in various natural language understanding tasks.
We propose EASE, an Explanation-Aware Soft Ensemble framework to empower in-context learning with LLMs.
arXiv Detail & Related papers (2023-11-13T06:13:38Z) - Interpreting Pretrained Language Models via Concept Bottlenecks [55.47515772358389]
Pretrained language models (PLMs) have made significant strides in various natural language processing tasks.
The lack of interpretability due to their black-box'' nature poses challenges for responsible implementation.
We propose a novel approach to interpreting PLMs by employing high-level, meaningful concepts that are easily understandable for humans.
arXiv Detail & Related papers (2023-11-08T20:41:18Z) - Adversarial Attacks on the Interpretation of Neuron Activation
Maximization [70.5472799454224]
Activation-maximization approaches are used to interpret and analyze trained deep-learning models.
In this work, we consider the concept of an adversary manipulating a model for the purpose of deceiving the interpretation.
arXiv Detail & Related papers (2023-06-12T19:54:33Z) - Learning with Explanation Constraints [91.23736536228485]
We provide a learning theoretic framework to analyze how explanations can improve the learning of our models.
We demonstrate the benefits of our approach over a large array of synthetic and real-world experiments.
arXiv Detail & Related papers (2023-03-25T15:06:47Z) - The Definitions of Interpretability and Learning of Interpretable Models [42.22982369082474]
We propose a mathematical definition for the human-interpretable model.
If a prediction model is interpretable by a human recognition system, the prediction model is defined as a completely human-interpretable model.
arXiv Detail & Related papers (2021-05-29T01:44:12Z) - Model Learning with Personalized Interpretability Estimation (ML-PIE) [2.862606936691229]
High-stakes applications require AI-generated models to be interpretable.
Current algorithms for the synthesis of potentially interpretable models rely on objectives or regularization terms.
We propose an approach for the synthesis of models that are tailored to the user.
arXiv Detail & Related papers (2021-04-13T09:47:48Z) - Interpretable Deep Learning: Interpretations, Interpretability,
Trustworthiness, and Beyond [49.93153180169685]
We introduce and clarify two basic concepts-interpretations and interpretability-that people usually get confused.
We elaborate the design of several recent interpretation algorithms, from different perspectives, through proposing a new taxonomy.
We summarize the existing work in evaluating models' interpretability using "trustworthy" interpretation algorithms.
arXiv Detail & Related papers (2021-03-19T08:40:30Z) - Efficient computation of contrastive explanations [8.132423340684568]
We study the relation of contrastive and counterfactual explanations.
We propose a 2-phase algorithm for efficiently computing (plausible) positives of many standard machine learning models.
arXiv Detail & Related papers (2020-10-06T11:50:28Z) - Interpretable Representations in Explainable AI: From Theory to Practice [7.031336702345381]
Interpretable representations are the backbone of many explainers that target black-box predictive systems.
We study properties of interpretable representations that encode presence and absence of human-comprehensible concepts.
arXiv Detail & Related papers (2020-08-16T21:44:03Z) - Multi-Objective Counterfactual Explanations [0.7349727826230864]
We propose the Multi-Objective Counterfactuals (MOC) method, which translates the counterfactual search into a multi-objective optimization problem.
Our approach not only returns a diverse set of counterfactuals with different trade-offs between the proposed objectives, but also maintains diversity in feature space.
arXiv Detail & Related papers (2020-04-23T13:56:39Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.