Models That Are Interpretable But Not Transparent

• URL: http://arxiv.org/abs/2502.19502v1
• Date: Wed, 26 Feb 2025 19:05:49 GMT
• Title: Models That Are Interpretable But Not Transparent
• Authors: Chudi Zhong, Panyu Chen, Cynthia Rudin,
• Abstract summary: FaithfulDefense creates explanations for logical models that are completely faithful, yet reveal as little as possible about the decision boundary.<n>This work provides an approach, FaithfulDefense, that creates model explanations for logical models that are completely faithful, yet reveal as little as possible about the decision boundary.
• Score: 19.6420087904074
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Faithful explanations are essential for machine learning models in high-stakes applications. Inherently interpretable models are well-suited for these applications because they naturally provide faithful explanations by revealing their decision logic. However, model designers often need to keep these models proprietary to maintain their value. This creates a tension: we need models that are interpretable--allowing human decision-makers to understand and justify predictions, but not transparent, so that the model's decision boundary is not easily replicated by attackers. Shielding the model's decision boundary is particularly challenging alongside the requirement of completely faithful explanations, since such explanations reveal the true logic of the model for an entire subspace around each query point. This work provides an approach, FaithfulDefense, that creates model explanations for logical models that are completely faithful, yet reveal as little as possible about the decision boundary. FaithfulDefense is based on a maximum set cover formulation, and we provide multiple formulations for it, taking advantage of submodularity.

Related papers

• Meaning Representations from Trajectories in Autoregressive Models [106.63181745054571]
  We propose to extract meaning representations from autoregressive language models by considering the distribution of all possible trajectories extending an input text. This strategy is prompt-free, does not require fine-tuning, and is applicable to any pre-trained autoregressive model. We empirically show that the representations obtained from large models align well with human annotations, outperform other zero-shot and prompt-free methods on semantic similarity tasks, and can be used to solve more complex entailment and containment tasks that standard embeddings cannot handle.
  arXiv  Detail & Related papers  (2023-10-23T04:35:58Z)
• Discriminative Feature Attributions: Bridging Post Hoc Explainability and Inherent Interpretability [29.459228981179674]
  Post hoc explanations incorrectly attribute high importance to features that are unimportant or non-discriminative for the underlying task. Inherently interpretable models, on the other hand, circumvent these issues by explicitly encoding explanations into model architecture. We propose Distractor Erasure Tuning (DiET), a method that adapts black-box models to be robust to distractor erasure.
  arXiv  Detail & Related papers  (2023-07-27T17:06:02Z)
• BELLA: Black box model Explanations by Local Linear Approximations [10.71090921516854]
  BELLA is a deterministic model-agnostic post-hoc approach for explaining the individual predictions of regression black-box models. BELLA can produce both factual and counterfactual explanations.
  arXiv  Detail & Related papers  (2023-05-18T21:22:23Z)
• Learning with Explanation Constraints [91.23736536228485]
  We provide a learning theoretic framework to analyze how explanations can improve the learning of our models. We demonstrate the benefits of our approach over a large array of synthetic and real-world experiments.
  arXiv  Detail & Related papers  (2023-03-25T15:06:47Z)
• Partial Order in Chaos: Consensus on Feature Attributions in the Rashomon Set [50.67431815647126]
  Post-hoc global/local feature attribution methods are being progressively employed to understand machine learning models. We show that partial orders of local/global feature importance arise from this methodology. We show that every relation among features present in these partial orders also holds in the rankings provided by existing approaches.
  arXiv  Detail & Related papers  (2021-10-26T02:53:14Z)
• Recurrence-Aware Long-Term Cognitive Network for Explainable Pattern Classification [0.0]
  We propose an LTCN-based model for interpretable pattern classification of structured data. Our method brings its own mechanism for providing explanations by quantifying the relevance of each feature in the decision process. Our interpretable model obtains competitive performance when compared to the state-of-the-art white and black boxes.
  arXiv  Detail & Related papers  (2021-07-07T18:14:50Z)
• A Holistic Approach to Interpretability in Financial Lending: Models, Visualizations, and Summary-Explanations [25.05825112699133]
  In a future world without such secrecy, what decision support tools would one want to use for justified lending decisions? We propose a framework for such decisions, including a globally interpretable machine learning model, an interactive visualization of it, and several types of summaries and explanations for any given decision. Our framework earned the FICO recognition award for the Explainable Machine Learning Challenge.
  arXiv  Detail & Related papers  (2021-06-04T17:05:25Z)
• An exact counterfactual-example-based approach to tree-ensemble models interpretability [0.0]
  High-performance models do not exhibit the necessary transparency to make their decisions fully understandable. We could derive an exact geometrical characterisation of their decision regions under the form of a collection of multidimensional intervals. An adaptation to reasoning on regression problems is also envisaged.
  arXiv  Detail & Related papers  (2021-05-31T09:32:46Z)
• Beyond Trivial Counterfactual Explanations with Diverse Valuable Explanations [64.85696493596821]
  In computer vision applications, generative counterfactual methods indicate how to perturb a model's input to change its prediction. We propose a counterfactual method that learns a perturbation in a disentangled latent space that is constrained using a diversity-enforcing loss. Our model improves the success rate of producing high-quality valuable explanations when compared to previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-03-18T12:57:34Z)
• Contrastive Explanations for Model Interpretability [77.92370750072831]
  We propose a methodology to produce contrastive explanations for classification models. Our method is based on projecting model representation to a latent space. Our findings shed light on the ability of label-contrastive explanations to provide a more accurate and finer-grained interpretability of a model's decision.
  arXiv  Detail & Related papers  (2021-03-02T00:36:45Z)
• The Struggles of Feature-Based Explanations: Shapley Values vs. Minimal Sufficient Subsets [61.66584140190247]
  We show that feature-based explanations pose problems even for explaining trivial models. We show that two popular classes of explainers, Shapley explainers and minimal sufficient subsets explainers, target fundamentally different types of ground-truth explanations.
  arXiv  Detail & Related papers  (2020-09-23T09:45:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.