On the Importance and Evaluation of Narrativity in Natural Language AI Explanations

• URL: http://arxiv.org/abs/2604.18311v1
• Date: Mon, 20 Apr 2026 14:17:39 GMT
• Title: On the Importance and Evaluation of Narrativity in Natural Language AI Explanations
• Abstract summary: We argue that XAI explanations should be presented in the form of narratives.<n> Narrative explanations support human understanding through four defining properties.<n>We propose seven automatic metrics that quantify the narrative quality of explanations.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Explainable AI (XAI) aims to make the behaviour of machine learning models interpretable, yet many explanation methods remain difficult to understand. The integration of Natural Language Generation into XAI aims to deliver explanations in textual form, making them more accessible to practitioners. Current approaches, however, largely yield static lists of feature importances. Although such explanations indicate what influences the prediction, they do not explain why the prediction occurs. In this study, we draw on insights from social sciences and linguistics, and argue that XAI explanations should be presented in the form of narratives. Narrative explanations support human understanding through four defining properties: continuous structure, cause-effect mechanisms, linguistic fluency, and lexical diversity. We show that standard Natural Language Processing (NLP) metrics based solely on token probability or word frequency fail to capture these properties and can be matched or exceeded by tautological text that conveys no explanatory content. To address this issue, we propose seven automatic metrics that quantify the narrative quality of explanations along the four identified dimensions. We benchmark current state-of-the-art explanation generation methods on six datasets and show that the proposed metrics separate descriptive from narrative explanations more reliably than standard NLP metrics. Finally, to further advance the field, we propose a set of problem-agnostic XAI Narrative generation rules for producing natural language XAI explanations, so that the resulting XAI Narratives exhibit stronger narrative properties and align with the findings from the linguistic and social science literature.

Related papers

• XAI-Grounded Explanation Generation for Speech Deepfake Detection with Training-Free Multimodal Large Language Models [48.03852608747844]
  Speech deepfake detection (SDD) systems require trustworthy explanations for reliable decision-making.<n>Traditional explainable AI (XAI) produces low-level attribution signals tightly coupled with model decisions.<n>Large language model (LLM)-based explanation generation produces generic and ungrounded descriptions.
  arXiv  Detail & Related papers  (2026-06-15T02:55:21Z)
• Explainability of Large Language Models: Opportunities and Challenges toward Generating Trustworthy Explanations [5.676319658620339]
  How a language model predicts the next token and generates content is not generally understandable by humans.<n>This paper investigates local explainability and mechanistic interpretability within Transformer-based large language models.
  arXiv  Detail & Related papers  (2025-10-20T07:43:53Z)
• XForecast: Evaluating Natural Language Explanations for Time Series Forecasting [72.57427992446698]
  Time series forecasting aids decision-making, especially for stakeholders who rely on accurate predictions. Traditional explainable AI (XAI) methods, which underline feature or temporal importance, often require expert knowledge. evaluating forecast NLEs is difficult due to the complex causal relationships in time series data.
  arXiv  Detail & Related papers  (2024-10-18T05:16:39Z)
• Natural Language Counterfactual Explanations for Graphs Using Large Language Models [7.560731917128082]
  We exploit the power of open-source Large Language Models to generate natural language explanations.<n>We show that our approach effectively produces accurate natural language representations of counterfactual instances.
  arXiv  Detail & Related papers  (2024-10-11T23:06:07Z)
• Explainability for Large Language Models: A Survey [59.67574757137078]
  Large language models (LLMs) have demonstrated impressive capabilities in natural language processing. This paper introduces a taxonomy of explainability techniques and provides a structured overview of methods for explaining Transformer-based language models.
  arXiv  Detail & Related papers  (2023-09-02T22:14:26Z)
• Natural Language Decompositions of Implicit Content Enable Better Text Representations [52.992875653864076]
  We introduce a method for the analysis of text that takes implicitly communicated content explicitly into account.<n>We use a large language model to produce sets of propositions that are inferentially related to the text that has been observed.<n>Our results suggest that modeling the meanings behind observed language, rather than the literal text alone, is a valuable direction for NLP.
  arXiv  Detail & Related papers  (2023-05-23T23:45:20Z)
• INTERACTION: A Generative XAI Framework for Natural Language Inference Explanations [58.062003028768636]
  Current XAI approaches only focus on delivering a single explanation. This paper proposes a generative XAI framework, INTERACTION (explaIn aNd predicT thEn queRy with contextuAl CondiTional varIational autO-eNcoder) Our novel framework presents explanation in two steps: (step one) Explanation and Label Prediction; and (step two) Diverse Evidence Generation.
  arXiv  Detail & Related papers  (2022-09-02T13:52:39Z)
• Scientific Explanation and Natural Language: A Unified Epistemological-Linguistic Perspective for Explainable AI [2.7920304852537536]
  This paper focuses on the scientific domain, aiming to bridge the gap between theory and practice on the notion of a scientific explanation. Through a mixture of quantitative and qualitative methodologies, the presented study allows deriving the following main conclusions.
  arXiv  Detail & Related papers  (2022-05-03T22:31:42Z)
• Towards Interpretable Natural Language Understanding with Explanations as Latent Variables [146.83882632854485]
  We develop a framework for interpretable natural language understanding that requires only a small set of human annotated explanations for training. Our framework treats natural language explanations as latent variables that model the underlying reasoning process of a neural model.
  arXiv  Detail & Related papers  (2020-10-24T02:05:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.