Automated Smell Detection and Recommendation in Natural Language Requirements

• URL: http://arxiv.org/abs/2305.07097v2
• Date: Sat, 25 Nov 2023 08:39:37 GMT
• Title: Automated Smell Detection and Recommendation in Natural Language Requirements
• Authors: Alvaro Veizaga, Seung Yeob Shin, Lionel C. Briand
• Abstract summary: Paska is a tool that takes as input any natural language (NL) requirements. It automatically detects quality problems as smells in the requirements, and offers recommendations to improve their quality.
• Score: 8.672583050502496
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Requirement specifications are typically written in natural language (NL) due to its usability across multiple domains and understandability by all stakeholders. However, unstructured NL is prone to quality problems (e.g., ambiguity) when writing requirements, which can result in project failures. To address this issue, we present a tool, named Paska, that takes as input any NL requirements, automatically detects quality problems as smells in the requirements, and offers recommendations to improve their quality. Our approach relies on natural language processing (NLP) techniques and a state-of-the-art controlled natural language (CNL) for requirements (Rimay), to detect smells and suggest recommendations using patterns defined in Rimay to improve requirement quality. We evaluated Paska through an industrial case study in the financial domain involving 13 systems and 2725 annotated requirements. The results show that our tool is accurate in detecting smells (89% precision and recall) and suggesting appropriate Rimay pattern recommendations (96% precision and 94% recall).

Related papers

• Can a Multichoice Dataset be Repurposed for Extractive Question Answering? [52.28197971066953]
  We repurposed the Belebele dataset (Bandarkar et al., 2023), which was designed for multiple-choice question answering (MCQA) We present annotation guidelines and a parallel EQA dataset for English and Modern Standard Arabic (MSA). Our aim is to enable others to adapt our approach for the 120+ other language variants in Belebele, many of which are deemed under-resourced.
  arXiv  Detail & Related papers  (2024-04-26T11:46:05Z)
• Requirements Satisfiability with In-Context Learning [1.747623282473278]
  Language models that can learn a task at an inference time, called in-context learning (ICL), show increasing promise in natural language tasks. In this paper, we apply ICL to a design evaluation of satisfaction arguments, which describe how a requirement is satisfied by a system specification and associated knowledge. The approach builds on three prompt design patterns, including augmented generation, prompt tuning, and chain-of-thought prompting.
  arXiv  Detail & Related papers  (2024-04-19T01:58:24Z)
• Natural Language Requirements Testability Measurement Based on Requirement Smells [1.1663475941322277]
  Testable requirements help prevent failures, reduce maintenance costs, and make it easier to perform acceptance tests. No automatic approach for measuring requirements testability has been proposed based on the requirements smells. This paper presents a mathematical model to evaluate and rank the natural language requirements testability based on an extensive set of nine requirements smells.
  arXiv  Detail & Related papers  (2024-03-26T08:19:29Z)
• Uncertainty-aware Language Modeling for Selective Question Answering [107.47864420630923]
  We present an automatic large language model (LLM) conversion approach that produces uncertainty-aware LLMs. Our approach is model- and data-agnostic, is computationally-efficient, and does not rely on external models or systems.
  arXiv  Detail & Related papers  (2023-11-26T22:47:54Z)
• Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models [54.55088169443828]
  Chain-of-Noting (CoN) is a novel approach aimed at improving the robustness of RALMs in facing noisy, irrelevant documents and in handling unknown scenarios. CoN achieves an average improvement of +7.9 in EM score given entirely noisy retrieved documents and +10.5 in rejection rates for real-time questions that fall outside the pre-training knowledge scope.
  arXiv  Detail & Related papers  (2023-11-15T18:54:53Z)
• CiRA: An Open-Source Python Package for Automated Generation of Test Case Descriptions from Natural Language Requirements [1.3082545468017672]
  This paper presents a tool from the CiRA (Causality In Requirements Artifacts) initiative, which automatically processes conditional natural language requirements. We evaluate the tool on a publicly available data set of 61 requirements from the requirements specification of the German Corona-Warn-App.
  arXiv  Detail & Related papers  (2023-10-12T11:30:59Z)
• Requirements Quality Assurance in Industry: Why, What and How? [3.6142643912711794]
  We propose a taxonomy of requirements quality assurance complexity that characterizes cognitive load of verifying a quality aspect from the human perspective. Once this taxonomy is realized and validated, it can serve as the basis for a decision framework of automated requirements quality assurance support.
  arXiv  Detail & Related papers  (2023-08-24T14:31:52Z)
• Privacy Adhering Machine Un-learning in NLP [66.17039929803933]
  In real world industry use Machine Learning to build models on user data. Such mandates require effort both in terms of data as well as model retraining. continuous removal of data and model retraining steps do not scale. We propose textitMachine Unlearning to tackle this challenge.
  arXiv  Detail & Related papers  (2022-12-19T16:06:45Z)
• Measuring Uncertainty in Translation Quality Evaluation (TQE) [62.997667081978825]
  This work carries out motivated research to correctly estimate the confidence intervals citeBrown_etal2001Interval depending on the sample size of the translated text. The methodology we applied for this work is from Bernoulli Statistical Distribution Modelling (BSDM) and Monte Carlo Sampling Analysis (MCSA)
  arXiv  Detail & Related papers  (2021-11-15T12:09:08Z)
• TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing [73.16475763422446]
  We propose a multilingual robustness evaluation platform for NLP tasks (TextFlint) It incorporates universal text transformation, task-specific transformation, adversarial attack, subpopulation, and their combinations to provide comprehensive robustness analysis. TextFlint generates complete analytical reports as well as targeted augmented data to address the shortcomings of the model's robustness.
  arXiv  Detail & Related papers  (2021-03-21T17:20:38Z)
• On Systematically Building a Controlled Natural Language for Functional Requirements [2.9676973500772887]
  Natural language (NL) is pervasive in software requirements specifications (SRSs) Despite its popularity and widespread use, NL is highly prone to quality issues such as vagueness, ambiguity, and incompleteness. Controlled natural languages (CNLs) have been proposed as a way to prevent quality problems in requirements documents.
  arXiv  Detail & Related papers  (2020-05-04T09:55:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.