Writing Better Software Explanations: A Guideline-Based Approach

• URL: http://arxiv.org/abs/2606.10880v1
• Date: Tue, 09 Jun 2026 13:54:24 GMT
• Title: Writing Better Software Explanations: A Guideline-Based Approach
• Abstract summary: We present a guideline-based formulation support tool for software explanations.<n>The tool structures the writing process into generation, quality checking, and iterative revision, while keeping domain control with developers.<n>Our findings suggest potential efficiency gains and higher perceived formulation quality through guideline-driven assistance.
• Score: 1.9218231341036756
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As software systems increasingly rely on natural-language explanations to address user-reported explanation needs in requirements communication and support, ensuring that such explanations are consistent, relevant, and well formulated remains a major challenge. Purely automatic large language model (LLM) generation often lacks reliable grounding and controllable output quality. In this paper, we present a guideline-based formulation support tool for software explanations that combines LLM-assisted text generation with an empirically derived quality guideline. The tool structures the writing process into generation, quality checking, and iterative revision, while keeping domain control with developers. We evaluated the approach in a two-phase study consisting of an interview-based developer experiment and a controlled user survey. Six industry practitioners with software development or DevOps experience formulated explanations for real explanation needs in a human-only manual condition and in a human-with-LLM-support condition. In this small-scale evaluation, tool-supported formulation was on average 24.4% faster, although inferential analyses indicated only a trend for efficiency. In a subsequent user study with 17 participants and 204 paired comparisons, tool-supported explanations were rated significantly higher in overall satisfaction than manual explanations (p=0.003, rank-biserial correlation=0.86). Our findings suggest potential efficiency gains and higher perceived formulation quality through guideline-driven LLM assistance. Future work should examine long-term industrial use and integration into existing development workflows.

Related papers

• Beyond the Traceback: Using LLMs for Adaptive Explanations of Programming Errors [52.619793817715326]
  We present a multi-stage crowdsourced study evaluating skill-targeted, LLM-generated Python error messages.<n>We show that while LLM-rewritten messages significantly improved subjective evaluations, these perceived gains did not translate into statistically significant improvements in objective debug performance.<n>This highlights a critical human-AI complementarity gap: explanations that feel better to users do not necessarily make them more effective debuggers.
  arXiv  Detail & Related papers  (2026-08-21T09:14:28Z)
• Exploring Dependence, Overreliance, and Addiction Related Behaviors Associated with Large Language Model Use Among Software Engineers [1.8757823231879849]
  This study investigates how software engineers experience the use of Large Language Models (LLMs) during professional software development.<n>The data were analyzed using descriptive statistics and qualitative thematic analysis of participants' open-ended responses.
  arXiv  Detail & Related papers  (2026-08-06T03:27:57Z)
• From Quality Properties to Practice: A Guideline and Workflow for Explainability Requirements [1.9218231341036756]
  Explainability is increasingly required in AI-enabled software systems.<n>Unguided large language model support can yield vague, inconsistent, or incomplete statements.<n>This paper presents a sequential, guideline-driven workflow for explainability requirements.
  arXiv  Detail & Related papers  (2026-06-09T13:56:31Z)
• Assistance to Autonomy: A Systematic Literature Review of Agentic AI across the Software Development Life Cycle [4.241377338556251]
  This study contributes a comprehensive characterization of the current literature on agentic systems in software product development.<n>It also provides a methodological contribution in the form of an AI-assisted tool to automate the screening phase in high-volume SLR domains.
  arXiv  Detail & Related papers  (2026-05-14T10:46:51Z)
• Prompt-Driven Code Summarization: A Systematic Literature Review [4.238464304669881]
  Large language models (LLMs) offer a promising solution by automatically generating natural language descriptions from source code.<n>This systematic review literature consolidates existing evidence, categorizes prompting paradigms, examines their effectiveness, and identifies gaps to guide future research and practical adoption.
  arXiv  Detail & Related papers  (2026-04-16T03:25:28Z)
• From Helpful to Trustworthy: LLM Agents for Pair Programming [0.0]
  LLM-based coding agents are increasingly used to generate code, tests, and documentation.<n>Still, their outputs can be plausible yet misaligned with developer intent and provide limited evidence for review in evolving projects.<n>This research proposes a systematic study of multi-agent LLM pair programming that externalizes intent and uses development tools for iterative validation.
  arXiv  Detail & Related papers  (2026-04-11T17:39:57Z)
• LoCoBench-Agent: An Interactive Benchmark for LLM Agents in Long-Context Software Engineering [90.84806758077536]
  We introduce textbfLoCoBench-Agent, a comprehensive evaluation framework specifically designed to assess large language models (LLMs) agents in realistic, long-context software engineering.<n>Our framework extends LoCoBench's 8,000 scenarios into interactive agent environments, enabling systematic evaluation of multi-turn conversations.<n>Our framework provides agents with 8 specialized tools (file operations, search, code analysis) and evaluates them across context lengths ranging from 10K to 1M tokens.
  arXiv  Detail & Related papers  (2025-11-17T23:57:24Z)
• Software Testing with Large Language Models: An Interview Study with Practitioners [2.198430261120653]
  The use of large language models in software testing is growing fast as they support numerous tasks.<n>However, their adoption often relies on informal experimentation rather than structured guidance.<n>This study investigates how software testing professionals use LLMs in practice to propose a preliminary, practitioner-informed guideline.
  arXiv  Detail & Related papers  (2025-10-20T05:06:56Z)
• How can we assess human-agent interactions? Case studies in software agent design [52.953425368394306]
  We make two major steps towards the rigorous assessment of human-agent interactions.<n>We propose PULSE, a framework for more efficient human-centric evaluation of agent designs.<n>We deploy the framework on a large-scale web platform built around the open-source software agent OpenHands.
  arXiv  Detail & Related papers  (2025-10-10T19:04:28Z)
• Improving Large Language Models Function Calling and Interpretability via Guided-Structured Templates [56.73907811047611]
  Large language models (LLMs) have demonstrated strong reasoning and tool-use capabilities.<n>LLMs often fail in real-world tool-interactions due to incorrect parameterization, poor tool selection, or misinterpretation of user intent.<n>We introduce a curriculum-inspired framework that leverages structured reasoning templates to guide LLMs through more deliberate step-by-step instructions for generating function callings.
  arXiv  Detail & Related papers  (2025-09-22T17:55:14Z)
• Training Language Models to Generate Quality Code with Program Analysis Feedback [66.0854002147103]
  Code generation with large language models (LLMs) is increasingly adopted in production but fails to ensure code quality.<n>We propose REAL, a reinforcement learning framework that incentivizes LLMs to generate production-quality code.
  arXiv  Detail & Related papers  (2025-05-28T17:57:47Z)
• IDA-Bench: Evaluating LLMs on Interactive Guided Data Analysis [60.32962597618861]
  IDA-Bench is a novel benchmark evaluating large language models in multi-round interactive scenarios.<n>Agent performance is judged by comparing its final numerical output to the human-derived baseline.<n>Even state-of-the-art coding agents (like Claude-3.7-thinking) succeed on 50% of the tasks, highlighting limitations not evident in single-turn tests.
  arXiv  Detail & Related papers  (2025-05-23T09:37:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.