From Quality Properties to Practice: A Guideline and Workflow for Explainability Requirements

• URL: http://arxiv.org/abs/2606.10882v1
• Date: Tue, 09 Jun 2026 13:56:31 GMT
• Title: From Quality Properties to Practice: A Guideline and Workflow for Explainability Requirements
• Abstract summary: Explainability is increasingly required in AI-enabled software systems.<n>Unguided large language model support can yield vague, inconsistent, or incomplete statements.<n>This paper presents a sequential, guideline-driven workflow for explainability requirements.
• Score: 1.9218231341036756
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Explainability is increasingly required in AI-enabled software systems to support transparency, user trust, and compliance. Yet, explainability requirements are often written ad hoc, and unguided large language model support can yield vague, inconsistent, or incomplete statements. This paper presents a sequential, guideline-driven workflow for formulating explainability requirements and evaluates its tool-based operationalization. We first elicited candidate quality properties through a structured literature review and developer interviews. We then prioritized these properties in an online survey with practitioners (n=20) and derived a concise guideline of ten core properties with actionable formulation instructions. Next, we operationalized the guideline in a web-based tool that supports an iterative workflow of drafting, property-based checks, and revision. We evaluated the workflow in two complementary studies. In a workshop with requirements engineers (n=6), tool support reduced formulation time by 23.5% on average (Wilcoxon p=0.021). In an independent online study with software developers (n=18), tool-supported and manually written requirements received comparable ratings for implementability and formulation quality, with a descriptive slight preference tendency toward the tool-supported versions. Overall, our results suggest that combining a prioritized quality guideline with lightweight LLM support can reduce formulation effort while producing requirements that are perceived comparably to manually written ones.

Related papers

• ClarifyCodeBench: Evaluating LLMs on Clarifying Ambiguous Requirements for Code Generation [54.788849448970154]
  We introduce ClarifyCodeBench, a novel interactive benchmark for evaluating Large Language Models' capability in resolving requirement ambiguity.<n>We conduct a systematic evaluation of six state-of-the-art LLMs using ClarifyCodeBench.<n>Our empirical results yield three critical insights: 1) Capability Decoupling: Strong code generation performance does not inherently translate to effective requirement clarification; 2) The Reasoning Paradox: While increased computational thinking enhances code correctness, it yields marginal gains in identifying ambiguities.
  arXiv  Detail & Related papers  (2026-07-01T09:58:59Z)
• Writing Better Software Explanations: A Guideline-Based Approach [1.9218231341036756]
  We present a guideline-based formulation support tool for software explanations.<n>The tool structures the writing process into generation, quality checking, and iterative revision, while keeping domain control with developers.<n>Our findings suggest potential efficiency gains and higher perceived formulation quality through guideline-driven assistance.
  arXiv  Detail & Related papers  (2026-06-09T13:54:24Z)
• Prompt-Driven Code Summarization: A Systematic Literature Review [4.238464304669881]
  Large language models (LLMs) offer a promising solution by automatically generating natural language descriptions from source code.<n>This systematic review literature consolidates existing evidence, categorizes prompting paradigms, examines their effectiveness, and identifies gaps to guide future research and practical adoption.
  arXiv  Detail & Related papers  (2026-04-16T03:25:28Z)
• Improving Large Language Models Function Calling and Interpretability via Guided-Structured Templates [56.73907811047611]
  Large language models (LLMs) have demonstrated strong reasoning and tool-use capabilities.<n>LLMs often fail in real-world tool-interactions due to incorrect parameterization, poor tool selection, or misinterpretation of user intent.<n>We introduce a curriculum-inspired framework that leverages structured reasoning templates to guide LLMs through more deliberate step-by-step instructions for generating function callings.
  arXiv  Detail & Related papers  (2025-09-22T17:55:14Z)
• Automatic Generation of Explainability Requirements and Software Explanations From User Reviews [2.2392379251177696]
  This work contributes to the advancement of explainability requirements in software systems by introducing an automated approach to derive requirements from user reviews and generate corresponding explanations.<n>We created a dataset of 58 user reviews, each annotated with manually crafted explainability requirements and explanations.<n>Our evaluation shows that while AI-generated requirements often lack relevance and correctness compared to human-created ones, the AI-generated explanations are frequently preferred for their clarity and style.
  arXiv  Detail & Related papers  (2025-07-10T00:03:36Z)
• Interactive Agents to Overcome Ambiguity in Software Engineering [61.40183840499932]
  AI agents are increasingly being deployed to automate tasks, often based on ambiguous and underspecified user instructions.<n>Making unwarranted assumptions and failing to ask clarifying questions can lead to suboptimal outcomes.<n>We study the ability of LLM agents to handle ambiguous instructions in interactive code generation settings by evaluating proprietary and open-weight models on their performance.
  arXiv  Detail & Related papers  (2025-02-18T17:12:26Z)
• Pointwise Mutual Information as a Performance Gauge for Retrieval-Augmented Generation [78.28197013467157]
  We show that the pointwise mutual information between a context and a question is an effective gauge for language model performance.<n>We propose two methods that use the pointwise mutual information between a document and a question as a gauge for selecting and constructing prompts that lead to better performance.
  arXiv  Detail & Related papers  (2024-11-12T13:14:09Z)
• CELA: Cost-Efficient Language Model Alignment for CTR Prediction [70.65910069412944]
  Click-Through Rate (CTR) prediction holds a paramount position in recommender systems.<n>Recent efforts have sought to mitigate these challenges by integrating Pre-trained Language Models (PLMs)<n>We propose textbfCost-textbfEfficient textbfLanguage Model textbfAlignment (textbfCELA) for CTR prediction.
  arXiv  Detail & Related papers  (2024-05-17T07:43:25Z)
• Learning Action Conditions from Instructional Manuals for Instruction Understanding [48.52663250368341]
  We propose a task dubbed action condition inference, and collecting a high-quality, human annotated dataset of preconditions and postconditions of actions in instructional manuals. We propose a weakly supervised approach to automatically construct large-scale training instances from online instructional manuals, and curate a densely human-annotated and validated dataset to study how well the current NLP models can infer action-condition dependencies in instruction texts.
  arXiv  Detail & Related papers  (2022-05-25T00:19:59Z)
• Natural Language in Requirements Engineering for Structure Inference -- An Integrative Review [0.0]
  The paper provides an integrative review regarding Natural Language Processing tools for Requirements Engineering. Results are that currently no open source approach exists that allows for the direct/primary extraction of information structure. An approach that allows for individual management of the algorithm, knowledge base, and text corpus is a possibility being pursued.
  arXiv  Detail & Related papers  (2022-02-10T14:46:09Z)
• Generating Fact Checking Explanations [52.879658637466605]
  A crucial piece of the puzzle that is still missing is to understand how to automate the most elaborate part of the process. This paper provides the first study of how these explanations can be generated automatically based on available claim context. Our results indicate that optimising both objectives at the same time, rather than training them separately, improves the performance of a fact checking system.
  arXiv  Detail & Related papers  (2020-04-13T05:23:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.