The Effect of Model Size on LLM Post-hoc Explainability via LIME

• URL: http://arxiv.org/abs/2405.05348v1
• Date: Wed, 8 May 2024 18:27:20 GMT
• Title: The Effect of Model Size on LLM Post-hoc Explainability via LIME
• Authors: Henning Heyen, Amy Widdicombe, Noah Y. Siegel, Maria Perez-Ortiz, Philip Treleaven,
• Abstract summary: This work explores LIME explanations for DeBERTaV3 models of four different sizes on natural language inference tasks. We evaluate the explanations based on their faithfulness to the models' internal decision processes and their plausibility. The key finding is that increased model size does not correlate with plausibility despite improved model performance.
• Score: 1.1073658091405039
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) are becoming bigger to boost performance. However, little is known about how explainability is affected by this trend. This work explores LIME explanations for DeBERTaV3 models of four different sizes on natural language inference (NLI) and zero-shot classification (ZSC) tasks. We evaluate the explanations based on their faithfulness to the models' internal decision processes and their plausibility, i.e. their agreement with human explanations. The key finding is that increased model size does not correlate with plausibility despite improved model performance, suggesting a misalignment between the LIME explanations and the models' internal processes as model size increases. Our results further suggest limitations regarding faithfulness metrics in NLI contexts.

Related papers

• "Why" Has the Least Side Effect on Model Editing [25.67779910446609]
  This paper delves into a critical factor-question type-by categorizing model editing questions. Our findings reveal that the extent of performance degradation varies significantly across different question types. We also examine the impact of batch size on side effects, discovering that increasing the batch size can mitigate performance drops.
  arXiv  Detail & Related papers  (2024-09-27T12:05:12Z)
• Evaluating the Reliability of Self-Explanations in Large Language Models [2.8894038270224867]
  We evaluate two kinds of such self-explanations - extractive and counterfactual. Our findings reveal, that, while these self-explanations can correlate with human judgement, they do not fully and accurately follow the model's decision process. We show that this gap can be bridged because prompting LLMs for counterfactual explanations can produce faithful, informative, and easy-to-verify results.
  arXiv  Detail & Related papers  (2024-07-19T17:41:08Z)
• DEAL: Disentangle and Localize Concept-level Explanations for VLMs [10.397502254316645]
  Large pre-trained Vision-Language Models might not be able to identify fine-grained concepts. We propose to DisEnt and Localize (Angle) concept-level explanations for concepts without human annotations. Our empirical results demonstrate that the proposed method significantly improves the concept-level explanations of the model in terms of disentanglability and localizability.
  arXiv  Detail & Related papers  (2024-07-19T15:39:19Z)
• Show Me How It's Done: The Role of Explanations in Fine-Tuning Language Models [0.45060992929802207]
  We show the significant benefits of using fine-tuning with explanations to enhance the performance of language models. We found that even smaller language models with as few as 60 million parameters benefited substantially from this approach.
  arXiv  Detail & Related papers  (2024-02-12T10:11:50Z)
• Explanation-aware Soft Ensemble Empowers Large Language Model In-context Learning [50.00090601424348]
  Large language models (LLMs) have shown remarkable capabilities in various natural language understanding tasks. We propose EASE, an Explanation-Aware Soft Ensemble framework to empower in-context learning with LLMs.
  arXiv  Detail & Related papers  (2023-11-13T06:13:38Z)
• Explainability for Large Language Models: A Survey [59.67574757137078]
  Large language models (LLMs) have demonstrated impressive capabilities in natural language processing. This paper introduces a taxonomy of explainability techniques and provides a structured overview of methods for explaining Transformer-based language models.
  arXiv  Detail & Related papers  (2023-09-02T22:14:26Z)
• Benchmarking Faithfulness: Towards Accurate Natural Language Explanations in Vision-Language Tasks [0.0]
  Natural language explanations (NLEs) promise to enable the communication of a model's decision-making in an easily intelligible way. While current models successfully generate convincing explanations, it is an open question how well the NLEs actually represent the reasoning process of the models. We propose three faithfulness metrics: Attribution-Similarity, NLE-Sufficiency, and NLE-Comprehensiveness.
  arXiv  Detail & Related papers  (2023-04-03T08:24:10Z)
• Large Language Models with Controllable Working Memory [64.71038763708161]
  Large language models (LLMs) have led to a series of breakthroughs in natural language processing (NLP) What further sets these models apart is the massive amounts of world knowledge they internalize during pretraining. How the model's world knowledge interacts with the factual information presented in the context remains under explored.
  arXiv  Detail & Related papers  (2022-11-09T18:58:29Z)
• Explanations from Large Language Models Make Small Reasoners Better [61.991772773700006]
  We show that our method can consistently and significantly outperform finetuning baselines across different settings. As a side benefit, human evaluation shows that our method can generate high-quality explanations to justify its predictions.
  arXiv  Detail & Related papers  (2022-10-13T04:50:02Z)
• To what extent do human explanations of model behavior align with actual model behavior? [91.67905128825402]
  We investigated the extent to which human-generated explanations of models' inference decisions align with how models actually make these decisions. We defined two alignment metrics that quantify how well natural language human explanations align with model sensitivity to input words. We find that a model's alignment with human explanations is not predicted by the model's accuracy on NLI.
  arXiv  Detail & Related papers  (2020-12-24T17:40:06Z)
• Leakage-Adjusted Simulatability: Can Models Generate Non-Trivial Explanations of Their Behavior in Natural Language? [86.60613602337246]
  We introduce a leakage-adjusted simulatability (LAS) metric for evaluating NL explanations. LAS measures how well explanations help an observer predict a model's output, while controlling for how explanations can directly leak the output. We frame explanation generation as a multi-agent game and optimize explanations for simulatability while penalizing label leakage.
  arXiv  Detail & Related papers  (2020-10-08T16:59:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.